{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5QNW8f3BPIoi"
   },
   "source": [
    "# Deep Learning with PyTorch Step-by-Step: A Beginner's Guide"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JQfcyrdyPIok"
   },
   "source": [
    "# Chapter 11"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AA_yhplzPIol"
   },
   "source": [
    "## Additional Setup\n",
    "\n",
    "The repository for the `allennlp` library has been archived. The library was used by `flair` to provide ELMO embeddings. This required PyTorch to be downgraded to version 1.7.1, and it prevented updating other libraries used in this chapter.\n",
    "\n",
    "At this point, the sensible solution is to upgrade `flair` and the other packages to more up-to-date versions and retire the use of ELMO embeddings. For the sake of completion, these embeddings will be replaced by flair embeddings in the code. Some code will be commented out whenever the replacement produces results that are substantially different from those produced by ELMO.\n",
    "\n",
    "Moreover, upgrading the `gensim` library required a few changes in the code since some attributes were changed from its 3.8.3 to its 4.3.3 version. These changes are highlighted in the code.\n",
    "\n",
    "**IMPORTANT**: due to the major version upgrade, the output of several cells will be quite different than those shown in the book. The new Gensim version produces slightly different vocabularies, so you should expect small changes in the number of words in the vocabulary, for example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "C8BhclloPIom",
    "outputId": "6b405ca2-f7de-45aa-feb5-e586bf1e9ef6"
   },
   "outputs": [],
   "source": [
    "# # UPDATED\n",
    "# ###########################################################\n",
    "# !pip install gensim==4.3.3\n",
    "# # The library has been archived and won't be used anymore\n",
    "# # # !pip install allennlp==0.9.0\n",
    "# !pip install flair==0.13.1\n",
    "# !pip install torchvision==0.18.1\n",
    "# # # HuggingFace\n",
    "# !pip install transformers==4.42.4\n",
    "# !pip install datasets==2.18.0\n",
    "# ###########################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "w5okKRDWPIon",
    "outputId": "93fc601d-5f4d-4f97-bcdb-eb7f5ea0a2db"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading files from GitHub repo to Colab...\n",
      "Finished!\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    import google.colab\n",
    "    import requests\n",
    "    url = 'https://raw.githubusercontent.com/dvgodoy/PyTorchStepByStep/master/config.py'\n",
    "    r = requests.get(url, allow_redirects=True)\n",
    "    open('config.py', 'wb').write(r.content)\n",
    "except ModuleNotFoundError:\n",
    "    pass\n",
    "\n",
    "from config import *\n",
    "config_chapter11()\n",
    "# This is needed to render the plots in this chapter\n",
    "from plots.chapter11 import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "MMmcGqLxPIoo"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import json\n",
    "import errno\n",
    "import requests\n",
    "import numpy as np\n",
    "from copy import deepcopy\n",
    "from operator import itemgetter\n",
    "\n",
    "import torch\n",
    "import torch.optim as optim\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "from torch.utils.data import DataLoader, TensorDataset, Dataset\n",
    "\n",
    "from data_generation.nlp import ALICE_URL, WIZARD_URL, download_text\n",
    "from stepbystep.v4 import StepByStep\n",
    "# These are the classes we built in Chapter 10\n",
    "from seq2seq import *\n",
    "\n",
    "import nltk\n",
    "from nltk.tokenize import sent_tokenize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "fU1AMangTB1o"
   },
   "outputs": [],
   "source": [
    "import gensim\n",
    "from gensim import corpora, downloader\n",
    "from gensim.parsing.preprocessing import *\n",
    "from gensim.utils import simple_preprocess\n",
    "from gensim.models import Word2Vec"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "L6HR2OpPTtEJ"
   },
   "outputs": [],
   "source": [
    "from flair.data import Sentence\n",
    "#from flair.embeddings import ELMoEmbeddings, WordEmbeddings, \\\n",
    "#    TransformerWordEmbeddings, TransformerDocumentEmbeddings\n",
    "from flair.embeddings import WordEmbeddings, \\\n",
    "    TransformerWordEmbeddings, TransformerDocumentEmbeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "sfXyVryWT2mM"
   },
   "outputs": [],
   "source": [
    "from datasets import load_dataset, Split\n",
    "from transformers import (\n",
    "    DataCollatorForLanguageModeling,\n",
    "    BertModel, BertTokenizer, BertForSequenceClassification,\n",
    "    DistilBertModel, DistilBertTokenizer,\n",
    "    DistilBertForSequenceClassification,\n",
    "    AutoModelForSequenceClassification,\n",
    "    AutoModel, AutoTokenizer, AutoModelForCausalLM,\n",
    "    Trainer, TrainingArguments, pipeline, TextClassificationPipeline\n",
    ")\n",
    "from transformers.pipelines import SUPPORTED_TASKS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aOMOKCI3PIop"
   },
   "source": [
    "# Down the Yellow Brick Rabbit Hole"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RG2bSGFtPIop"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/alice_dorothy.png?raw=1)\n",
    "\n",
    "*Left: \"Alice and the Baby Pig\" illustration by John Tenniel's, from \"Alice's Adventure's in Wonderland\" (1865).*\n",
    "\n",
    "*Right: \"Dorothy meets the Cowardly Lion\" illustration by W.W. Denslow, from \"The Wonderful Wizard of Oz\" (1900)*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zqGeN0zzPIoq"
   },
   "source": [
    "# Building a Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "YIiNrb9uPIor"
   },
   "outputs": [],
   "source": [
    "localfolder = 'texts'\n",
    "download_text(ALICE_URL, localfolder)\n",
    "download_text(WIZARD_URL, localfolder)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "4IceFMuyPIor"
   },
   "outputs": [],
   "source": [
    "with open(os.path.join(localfolder, 'alice28-1476.txt'), 'r') as f:\n",
    "    alice = ''.join(f.readlines()[104:3704])\n",
    "\n",
    "with open(os.path.join(localfolder, 'wizoz10-1740.txt'), 'r') as f:\n",
    "    wizard = ''.join(f.readlines()[310:5100])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Hpw53y4yPIos",
    "outputId": "64da19ba-f1dd-418c-db6b-9903060884a2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                ALICE'S ADVENTURES IN WONDERLAND\n",
      "\n",
      "                          Lewis Carroll\n",
      "\n",
      "               THE MILLENNIUM FULCRUM EDITION 2.8\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "                            CHAPTER I\n",
      "\n",
      "                      Down the Rabbit-Hole\n",
      "\n",
      "\n",
      "  Alice was beginning to get very tired of sitting by her sister\n",
      "on the bank, and of having nothing to do:  once or twice she had\n",
      "peeped into the book her sister was reading, but it had no\n",
      "pictures or conversations in it, `and what is the use of a book,'\n",
      "thought Alice `w\n",
      "\n",
      "\n",
      "                    THE WONDERFUL WIZARD OF OZ\n",
      "\n",
      "\n",
      "                          1.  The Cyclone\n",
      "\n",
      "\n",
      "    Dorothy lived in the midst of the great Kansas prairies, with\n",
      "Uncle Henry, who was a farmer, and Aunt Em, who was the farmer's\n",
      "wife.  Their house was small, for the lumber to build it had to be\n",
      "carried by wagon many miles.  There were four walls, a floor and a\n",
      "roof, which made one room; and this room contained a rusty looking\n",
      "cookstove, a cupboard for the dishes, a table, three or four\n",
      "chairs, and th\n"
     ]
    }
   ],
   "source": [
    "print(alice[:500])\n",
    "print('\\n')\n",
    "print(wizard[:500])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "JNdQPUzqPIos"
   },
   "outputs": [],
   "source": [
    "text_cfg = \"\"\"fname,start,end\n",
    "alice28-1476.txt,104,3704\n",
    "wizoz10-1740.txt,310,5100\"\"\"\n",
    "bytes_written = open(os.path.join(localfolder, 'lines.cfg'), 'w').write(text_cfg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Cd4tQSexPIot"
   },
   "source": [
    "## Sentence Tokenization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Ms_jPDTWPIot",
    "outputId": "9ef3449a-d05b-448a-fa91-3e9039cf7379"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"I'm\", 'following', 'the', 'white', 'rabbit']"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence = \"I'm following the white rabbit\"\n",
    "tokens = sentence.split(' ')\n",
    "tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UxbboLMMPIou",
    "outputId": "aba0b3d4-fde7-44b9-bf4b-c18e3bcfe620"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
      "[nltk_data]   Unzipping tokenizers/punkt.zip.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(1612, 2240)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "from nltk.tokenize import sent_tokenize\n",
    "\n",
    "nltk.download('punkt')\n",
    "corpus_alice = sent_tokenize(alice)\n",
    "corpus_wizard = sent_tokenize(wizard)\n",
    "len(corpus_alice), len(corpus_wizard)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "IQ0TkRtRPIou",
    "outputId": "9d754535-e54c-4383-cc79-ec4e83734923"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      },
      "text/plain": [
       "'There was nothing so VERY remarkable in that; nor did Alice\\nthink it so VERY much out of the way to hear the Rabbit say to\\nitself, `Oh dear!'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corpus_alice[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "gFOXRwwLPIov",
    "outputId": "d632a88e-d682-4fd5-e752-4ae6da731ec9"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      },
      "text/plain": [
       "'\"There\\'s a cyclone coming, Em,\" he called to his wife.'"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corpus_wizard[30]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "8TX0L1NqPIov"
   },
   "outputs": [],
   "source": [
    "def sentence_tokenize(source, quote_char='\\\\', sep_char=',',\n",
    "                      include_header=True, include_source=True,\n",
    "                      extensions=('txt'), **kwargs):\n",
    "    nltk.download('punkt')\n",
    "    # If source is a folder, goes through all files inside it\n",
    "    # that match the desired extensions ('txt' by default)\n",
    "    if os.path.isdir(source):\n",
    "        filenames = [f for f in os.listdir(source)\n",
    "                     if os.path.isfile(os.path.join(source, f)) and\n",
    "                        os.path.splitext(f)[1][1:] in extensions]\n",
    "    elif isinstance(source, str):\n",
    "        filenames = [source]\n",
    "\n",
    "    # If there is a configuration file, builds a dictionary with\n",
    "    # the corresponding start and end lines of each text file\n",
    "    config_file = os.path.join(source, 'lines.cfg')\n",
    "    config = {}\n",
    "    if os.path.exists(config_file):\n",
    "        with open(config_file, 'r') as f:\n",
    "            rows = f.readlines()\n",
    "\n",
    "        for r in rows[1:]:\n",
    "            fname, start, end = r.strip().split(',')\n",
    "            config.update({fname: (int(start), int(end))})\n",
    "\n",
    "    new_fnames = []\n",
    "    # For each file of text\n",
    "    for fname in filenames:\n",
    "        # If there's a start and end line for that file, use it\n",
    "        try:\n",
    "            start, end = config[fname]\n",
    "        except KeyError:\n",
    "            start = None\n",
    "            end = None\n",
    "\n",
    "        # Opens the file, slices the configures lines (if any)\n",
    "        # cleans line breaks and uses the sentence tokenizer\n",
    "        with open(os.path.join(source, fname), 'r') as f:\n",
    "            contents = (''.join(f.readlines()[slice(start, end, None)])\n",
    "                        .replace('\\n', ' ').replace('\\r', ''))\n",
    "        corpus = sent_tokenize(contents, **kwargs)\n",
    "\n",
    "        # Builds a CSV file containing tokenized sentences\n",
    "        base = os.path.splitext(fname)[0]\n",
    "        new_fname = f'{base}.sent.csv'\n",
    "        new_fname = os.path.join(source, new_fname)\n",
    "        with open(new_fname, 'w') as f:\n",
    "            # Header of the file\n",
    "            if include_header:\n",
    "                if include_source:\n",
    "                    f.write('sentence,source\\n')\n",
    "                else:\n",
    "                    f.write('sentence\\n')\n",
    "            # Writes one line for each sentence\n",
    "            for sentence in corpus:\n",
    "                if include_source:\n",
    "                    f.write(f'{quote_char}{sentence}{quote_char}{sep_char}{fname}\\n')\n",
    "                else:\n",
    "                    f.write(f'{quote_char}{sentence}{quote_char}\\n')\n",
    "        new_fnames.append(new_fname)\n",
    "\n",
    "    # Returns list of the newly generated CSV files\n",
    "    return sorted(new_fnames)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "duMi-DzQPIov",
    "outputId": "923135e1-992c-45ea-ec79-2d964fae2b19"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
      "[nltk_data]   Package punkt is already up-to-date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['texts/alice28-1476.sent.csv', 'texts/wizoz10-1740.sent.csv']"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_fnames = sentence_tokenize(localfolder)\n",
    "new_fnames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "tvQ5ikWQPIow"
   },
   "outputs": [],
   "source": [
    "# import spacy\n",
    "# nlp = spacy.blank(\"en\")\n",
    "# nlp.add_pipe(nlp.create_pipe(\"sentencizer\"))\n",
    "\n",
    "# sentences = []\n",
    "# for doc in nlp.pipe(corpus_alice):\n",
    "#     sentences.extend(sent.text for sent in doc.sents)\n",
    "\n",
    "# len(sentences), sentences[2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TNMT7-ClPIow"
   },
   "source": [
    "## HuggingFace's Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s5T31ZJtPIow"
   },
   "source": [
    "## Loading a Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 49,
     "referenced_widgets": [
      "cd067f60eb7c45bf8010431d51b2fb34",
      "b89907d757a74899a7606c3b6ce383e5",
      "926dd4b6c57c42b68c44e06cb5013129",
      "9996784c4aa04ca480a58e53e5e073e5",
      "abaa724f6e984ddcb538a24012a74dd9",
      "49948cdf2ac94f45aa9c4a17c5996a17",
      "0c78bace44154985ad12c0265ae430f7",
      "ead4f1dddc0d48499955d55b069e82e0",
      "5d97ee42cb5b4ca5a9f5bc556f1a8a60",
      "a23cea85c11d44f99338ee370d27bb27",
      "e4a5b2625bfc4f7eb9cf1000c8fcd9fb"
     ]
    },
    "id": "6mmR-5ItPIox",
    "outputId": "704ce450-5933-4d75-86b8-ea7b8c60ea10"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "cd067f60eb7c45bf8010431d51b2fb34",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating train split: 0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from datasets import load_dataset, Split\n",
    "\n",
    "dataset = load_dataset(path='csv', data_files=new_fnames, quotechar='\\\\', split=Split.TRAIN)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RHikb18aPIox"
   },
   "source": [
    "### Attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "DvVIqWAiPIox",
    "outputId": "150d6181-5d50-4a47-f119-89348dd38995"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "({'sentence': Value(dtype='string', id=None),\n",
       "  'source': Value(dtype='string', id=None)},\n",
       " 2,\n",
       " (3852, 2))"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset.features, dataset.num_columns, dataset.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "MtH_c0gVPIoy",
    "outputId": "eaafb6da-0878-4343-a04f-27a0e9269dc4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'sentence': 'There was nothing so VERY remarkable in that; nor did Alice think it so VERY much out of the way to hear the Rabbit say to itself, `Oh dear!',\n",
       " 'source': 'alice28-1476.txt'}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xDOC9Hh4PIoy",
    "outputId": "3b226567-5116-429f-965c-533b5993ab21"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['alice28-1476.txt', 'alice28-1476.txt', 'alice28-1476.txt']"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset['source'][:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EM-mEXGrPIoy"
   },
   "source": [
    "### Methods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "QprAteUMPIoy",
    "outputId": "37b3a36c-1f14-40f4-c81f-3d2cafaf84cc"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['alice28-1476.txt', 'wizoz10-1740.txt']"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset.unique('source')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 49,
     "referenced_widgets": [
      "c60c89940e5e455bab9c7cf1e420d55b",
      "5f174bf3a8d94b51be76f7455a6621f0",
      "d0db897511f0440588131cd7621ab41b",
      "a40ffd8b51784c2988e75903ac4fab93",
      "5ac8c023798949aa85008138c59e5cbd",
      "f123f91546064a538a74c83d893d52df",
      "01ddfefb7f3e4a3589efab6d307a1bf0",
      "8aed6812b77f4175a7ba713d075b6db7",
      "c43c28578be8437ab5457b444b4103fd",
      "fc19e87c5a9b409e8877022912b3b5d4",
      "d747d7a4a9ac4e479e2cafb7ca11d0ad"
     ]
    },
    "id": "9XbmJVItPIoz",
    "outputId": "6abd3afa-fc5f-4f3f-ae86-9ea0face5747"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c60c89940e5e455bab9c7cf1e420d55b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/3852 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def is_alice_label(row):\n",
    "    is_alice = int(row['source'] == 'alice28-1476.txt')\n",
    "    return {'labels': is_alice}\n",
    "\n",
    "dataset = dataset.map(is_alice_label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Ei4rMiW0PIoz",
    "outputId": "daee5015-ff9e-457a-8b0a-e750430aa5de"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'sentence': 'There was nothing so VERY remarkable in that; nor did Alice think it so VERY much out of the way to hear the Rabbit say to itself, `Oh dear!',\n",
       " 'source': 'alice28-1476.txt',\n",
       " 'labels': 1}"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "id": "zL8_4qImPIo0"
   },
   "outputs": [],
   "source": [
    "shuffled_dataset = dataset.shuffle(seed=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "svODbO3NPIo0",
    "outputId": "b5c5b3a7-d7e9-42e5-ae00-50c797486cce"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatasetDict({\n",
       "    train: Dataset({\n",
       "        features: ['sentence', 'source', 'labels'],\n",
       "        num_rows: 3081\n",
       "    })\n",
       "    test: Dataset({\n",
       "        features: ['sentence', 'source', 'labels'],\n",
       "        num_rows: 771\n",
       "    })\n",
       "})"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "split_dataset = shuffled_dataset.train_test_split(test_size=0.2)\n",
    "split_dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "id": "a-vhKAPsPIo0"
   },
   "outputs": [],
   "source": [
    "train_dataset, test_dataset = split_dataset['train'], split_dataset['test']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jnTeb1APPIo0"
   },
   "source": [
    "## Data Augmentation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "v18lfIROPIo1"
   },
   "outputs": [],
   "source": [
    "# !pip install textattack\n",
    "# from textattack.augmentation import EmbeddingAugmenter\n",
    "# augmenter = EmbeddingAugmenter()\n",
    "# feynman = 'What I cannot create, I do not understand.'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "oKSTr__FPIo1"
   },
   "outputs": [],
   "source": [
    "# for i in range(5):\n",
    "#     print(augmenter.augment(feynman))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1Apyc3vyPIo1"
   },
   "source": [
    "# Word Tokenization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "H6C3Qto1PIo1",
    "outputId": "6b718179-290e-4ff6-b338-f11c0c616602"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"I'm\", 'following', 'the', 'white', 'rabbit']"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence = \"I'm following the white rabbit\"\n",
    "tokens = sentence.split(' ')\n",
    "tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "gMiBbirCPIo2",
    "outputId": "6d0b8691-1e88-4589-acec-ef9f59fd26b5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['follow', 'white', 'rabbit']"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from gensim.parsing.preprocessing import *\n",
    "\n",
    "preprocess_string(sentence)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "g9I0NFCcPIo2",
    "outputId": "39a7ae3e-0e40-4867-b5ed-def718032b3d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['i', 'm', 'following', 'the', 'white', 'rabbit']"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filters = [lambda x: x.lower(), strip_tags, strip_punctuation, strip_multiple_whitespaces, strip_numeric]\n",
    "preprocess_string(sentence, filters=filters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "I6zIZlDPPIo2",
    "outputId": "8b477950-1be3-4be5-93d8-9cc4ce2b0580"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['following', 'the', 'white', 'rabbit']"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from gensim.utils import simple_preprocess\n",
    "\n",
    "tokens = simple_preprocess(sentence)\n",
    "tokens"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "S41E3OspPIo3"
   },
   "source": [
    "## Vocabulary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Nj8VJgZ-PIo3",
    "outputId": "49b780c1-57ec-4ee6-b06b-90471554fe50"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['so',\n",
       " 'the',\n",
       " 'woodman',\n",
       " 'went',\n",
       " 'at',\n",
       " 'once',\n",
       " 'to',\n",
       " 'the',\n",
       " 'trees',\n",
       " 'and',\n",
       " 'began',\n",
       " 'to',\n",
       " 'work',\n",
       " 'and',\n",
       " 'he',\n",
       " 'soon',\n",
       " 'made',\n",
       " 'truck',\n",
       " 'out',\n",
       " 'of',\n",
       " 'the',\n",
       " 'limbs',\n",
       " 'of',\n",
       " 'trees',\n",
       " 'from',\n",
       " 'which',\n",
       " 'he',\n",
       " 'chopped',\n",
       " 'away',\n",
       " 'all',\n",
       " 'the',\n",
       " 'leaves',\n",
       " 'and',\n",
       " 'branches']"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentences = train_dataset['sentence']\n",
    "tokens = [simple_preprocess(sent) for sent in sentences]\n",
    "tokens[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "RmGTVyKkPIo3",
    "outputId": "1f5dc971-7166-435f-b343-0b6db1e39fa9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dictionary<3726 unique tokens: ['all', 'and', 'at', 'away', 'began']...>\n"
     ]
    }
   ],
   "source": [
    "from gensim import corpora\n",
    "\n",
    "dictionary = corpora.Dictionary(tokens)\n",
    "print(dictionary)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "TbIfKFBIPIo3",
    "outputId": "2b9b8e90-f89f-4c93-951b-112a9f90e310"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3081"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dictionary.num_docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fOPK8ULTPIo4",
    "outputId": "e323d08b-277c-49ca-b208-79c1cbc7bc4c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "50926"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dictionary.num_pos # processed words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3cR7uPunPIo4",
    "outputId": "84cfe9f1-fec3-4cd2-ebaf-582f56ae0223",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'all': 0,\n",
       " 'and': 1,\n",
       " 'at': 2,\n",
       " 'away': 3,\n",
       " 'began': 4,\n",
       " 'branches': 5,\n",
       " 'chopped': 6,\n",
       " 'from': 7,\n",
       " 'he': 8,\n",
       " 'leaves': 9,\n",
       " 'limbs': 10,\n",
       " 'made': 11,\n",
       " 'of': 12,\n",
       " 'once': 13,\n",
       " 'out': 14,\n",
       " 'so': 15,\n",
       " 'soon': 16,\n",
       " 'the': 17,\n",
       " 'to': 18,\n",
       " 'trees': 19,\n",
       " 'truck': 20,\n",
       " 'went': 21,\n",
       " 'which': 22,\n",
       " 'woodman': 23,\n",
       " 'work': 24,\n",
       " 'because': 25,\n",
       " 'have': 26,\n",
       " 'killed': 27,\n",
       " 'shoes': 28,\n",
       " 'silver': 29,\n",
       " 'wear': 30,\n",
       " 'wicked': 31,\n",
       " 'witch': 32,\n",
       " 'you': 33,\n",
       " 'anxiously': 34,\n",
       " 'asked': 35,\n",
       " 'girl': 36,\n",
       " 'are': 37,\n",
       " 'little': 38,\n",
       " 'man': 39,\n",
       " 'meekly': 40,\n",
       " 'no': 41,\n",
       " 'said': 42,\n",
       " 'wrong': 43,\n",
       " 'alice': 44,\n",
       " 'ask': 45,\n",
       " 'ventured': 46,\n",
       " 'about': 47,\n",
       " 'among': 48,\n",
       " 'as': 49,\n",
       " 'but': 50,\n",
       " 'distant': 51,\n",
       " 'except': 52,\n",
       " 'follow': 53,\n",
       " 'green': 54,\n",
       " 'moving': 55,\n",
       " 'result': 56,\n",
       " 'seemed': 57,\n",
       " 'shaking': 58,\n",
       " 'she': 59,\n",
       " 'spoke': 60,\n",
       " 'them': 61,\n",
       " 'was': 62,\n",
       " 'by': 63,\n",
       " 'full': 64,\n",
       " 'grown': 65,\n",
       " 'had': 66,\n",
       " 'her': 67,\n",
       " 'size': 68,\n",
       " 'this': 69,\n",
       " 'time': 70,\n",
       " 'again': 71,\n",
       " 'back': 72,\n",
       " 'be': 73,\n",
       " 'circus': 74,\n",
       " 'go': 75,\n",
       " 'in': 76,\n",
       " 'kansas': 77,\n",
       " 'much': 78,\n",
       " 'rather': 79,\n",
       " 'with': 80,\n",
       " 'bite': 81,\n",
       " 'come': 82,\n",
       " 'if': 83,\n",
       " 'will': 84,\n",
       " 'yard': 85,\n",
       " 'herself': 86,\n",
       " 'thought': 87,\n",
       " 'absence': 88,\n",
       " 'advantage': 89,\n",
       " 'cost': 90,\n",
       " 'delay': 91,\n",
       " 'game': 92,\n",
       " 'guests': 93,\n",
       " 'however': 94,\n",
       " 'hurried': 95,\n",
       " 'lives': 96,\n",
       " 'merely': 97,\n",
       " 'moment': 98,\n",
       " 'other': 99,\n",
       " 'queen': 100,\n",
       " 'remarking': 101,\n",
       " 'resting': 102,\n",
       " 'saw': 103,\n",
       " 'shade': 104,\n",
       " 'taken': 105,\n",
       " 'that': 106,\n",
       " 'their': 107,\n",
       " 'they': 108,\n",
       " 'were': 109,\n",
       " 'would': 110,\n",
       " 'before': 111,\n",
       " 'course': 112,\n",
       " 'each': 113,\n",
       " 'expected': 114,\n",
       " 'greatly': 115,\n",
       " 'looked': 116,\n",
       " 'one': 117,\n",
       " 'room': 118,\n",
       " 'see': 119,\n",
       " 'shape': 120,\n",
       " 'surprised': 121,\n",
       " 'when': 122,\n",
       " 'wizard': 123,\n",
       " 'am': 124,\n",
       " 'dorothy': 125,\n",
       " 'get': 126,\n",
       " 'how': 127,\n",
       " 'now': 128,\n",
       " 'bath': 129,\n",
       " 'child': 130,\n",
       " 'never': 131,\n",
       " 'night': 132,\n",
       " 'off': 133,\n",
       " 'pretty': 134,\n",
       " 'proud': 135,\n",
       " 'took': 136,\n",
       " 'could': 137,\n",
       " 'everywhere': 138,\n",
       " 'eye': 139,\n",
       " 'powerful': 140,\n",
       " 'telescope': 141,\n",
       " 'west': 142,\n",
       " 'yet': 143,\n",
       " 'brains': 144,\n",
       " 'decided': 145,\n",
       " 'hard': 146,\n",
       " 'his': 147,\n",
       " 'needles': 148,\n",
       " 'pins': 149,\n",
       " 'scarecrow': 150,\n",
       " 'stick': 151,\n",
       " 'think': 152,\n",
       " 'band': 153,\n",
       " 'beside': 154,\n",
       " 'brought': 155,\n",
       " 'flew': 156,\n",
       " 'into': 157,\n",
       " 'it': 158,\n",
       " 'magic': 159,\n",
       " 'monkeys': 160,\n",
       " 'open': 161,\n",
       " 'stood': 162,\n",
       " 'throne': 163,\n",
       " 'through': 164,\n",
       " 'window': 165,\n",
       " 'winged': 166,\n",
       " 'words': 167,\n",
       " 'an': 168,\n",
       " 'awfully': 169,\n",
       " 'balanced': 170,\n",
       " 'clever': 171,\n",
       " 'eel': 172,\n",
       " 'end': 173,\n",
       " 'ever': 174,\n",
       " 'hardly': 175,\n",
       " 'nose': 176,\n",
       " 'old': 177,\n",
       " 'on': 178,\n",
       " 'steady': 179,\n",
       " 'suppose': 180,\n",
       " 'what': 181,\n",
       " 'your': 182,\n",
       " 'youth': 183,\n",
       " 'after': 184,\n",
       " 'beat': 185,\n",
       " 'castle': 186,\n",
       " 'do': 187,\n",
       " 'down': 188,\n",
       " 'next': 189,\n",
       " 'returned': 190,\n",
       " 'sat': 191,\n",
       " 'sent': 192,\n",
       " 'should': 193,\n",
       " 'strap': 194,\n",
       " 'well': 195,\n",
       " 'dormouse': 196,\n",
       " 'shall': 197,\n",
       " 'then': 198,\n",
       " 'baked': 199,\n",
       " 'cracks': 200,\n",
       " 'gray': 201,\n",
       " 'land': 202,\n",
       " 'mass': 203,\n",
       " 'plowed': 204,\n",
       " 'running': 205,\n",
       " 'sun': 206,\n",
       " 'days': 207,\n",
       " 'two': 208,\n",
       " 'afraid': 209,\n",
       " 'coming': 210,\n",
       " 'heard': 211,\n",
       " 'like': 212,\n",
       " 'something': 213,\n",
       " 've': 214,\n",
       " 'cannot': 215,\n",
       " 'disease': 216,\n",
       " 'for': 217,\n",
       " 'heart': 218,\n",
       " 'my': 219,\n",
       " 'part': 220,\n",
       " 'screamed': 221,\n",
       " 'answered': 222,\n",
       " 'lighted': 223,\n",
       " 'match': 224,\n",
       " 'anything': 225,\n",
       " 'ate': 226,\n",
       " 'fed': 227,\n",
       " 'live': 228,\n",
       " 'neither': 229,\n",
       " 'nor': 230,\n",
       " 'not': 231,\n",
       " 'straw': 232,\n",
       " 'sure': 233,\n",
       " 'tin': 234,\n",
       " 'unless': 235,\n",
       " 'nothing': 236,\n",
       " 'proves': 237,\n",
       " 'sort': 238,\n",
       " 'both': 239,\n",
       " 'duchess': 240,\n",
       " 'flamingoes': 241,\n",
       " 'mustard': 242,\n",
       " 'true': 243,\n",
       " 'very': 244,\n",
       " 'being': 245,\n",
       " 'dear': 246,\n",
       " 'else': 247,\n",
       " 'first': 248,\n",
       " 'here': 249,\n",
       " 'll': 250,\n",
       " 'me': 251,\n",
       " 'oh': 252,\n",
       " 'person': 253,\n",
       " 'somebody': 254,\n",
       " 'stay': 255,\n",
       " 'tell': 256,\n",
       " 'till': 257,\n",
       " 'up': 258,\n",
       " 'knows': 259,\n",
       " 'such': 260,\n",
       " 'called': 261,\n",
       " 'close': 262,\n",
       " 'last': 263,\n",
       " 'third': 264,\n",
       " 'us': 265,\n",
       " 'uglifying': 266,\n",
       " 'changed': 267,\n",
       " 'too': 268,\n",
       " 'wind': 269,\n",
       " 'bye': 270,\n",
       " 'can': 271,\n",
       " 'good': 272,\n",
       " 'is': 273,\n",
       " 'leader': 274,\n",
       " 'luck': 275,\n",
       " 'summon': 276,\n",
       " 'getting': 277,\n",
       " 'happening': 278,\n",
       " 'queer': 279,\n",
       " 'things': 280,\n",
       " 'used': 281,\n",
       " 'arches': 282,\n",
       " 'caught': 283,\n",
       " 'doesn': 284,\n",
       " 'fight': 285,\n",
       " 'flamingo': 286,\n",
       " 'gone': 287,\n",
       " 'ground': 288,\n",
       " 'hedgehogs': 289,\n",
       " 'matter': 290,\n",
       " 'over': 291,\n",
       " 'side': 292,\n",
       " 'sight': 293,\n",
       " 'really': 294,\n",
       " 'deal': 295,\n",
       " 'great': 296,\n",
       " 'there': 297,\n",
       " 'alone': 298,\n",
       " 'executions': 299,\n",
       " 'gryphon': 300,\n",
       " 'leaving': 301,\n",
       " 'must': 302,\n",
       " 'ordered': 303,\n",
       " 'some': 304,\n",
       " 'walked': 305,\n",
       " 'any': 306,\n",
       " 'roof': 307,\n",
       " 'sense': 308,\n",
       " 'take': 309,\n",
       " 'makes': 310,\n",
       " 'mean': 311,\n",
       " 'shiny': 312,\n",
       " 'day': 313,\n",
       " 'fine': 314,\n",
       " 'carry': 315,\n",
       " 'consider': 316,\n",
       " 'desert': 317,\n",
       " 'find': 318,\n",
       " 'give': 319,\n",
       " 'or': 320,\n",
       " 'three': 321,\n",
       " 'try': 322,\n",
       " 'way': 323,\n",
       " 'angry': 324,\n",
       " 'came': 325,\n",
       " 'pat': 326,\n",
       " 'rabbit': 327,\n",
       " 'voice': 328,\n",
       " 'capital': 329,\n",
       " 'catching': 330,\n",
       " 'mice': 331,\n",
       " 'business': 332,\n",
       " 'know': 333,\n",
       " 'broken': 334,\n",
       " 'glass': 335,\n",
       " 'more': 336,\n",
       " 'sounds': 337,\n",
       " 'beg': 338,\n",
       " 'pardon': 339,\n",
       " 'crawled': 340,\n",
       " 'got': 341,\n",
       " 'grass': 342,\n",
       " 'grow': 343,\n",
       " 'make': 344,\n",
       " 'mushroom': 345,\n",
       " 'shorter': 346,\n",
       " 'taller': 347,\n",
       " 'bent': 348,\n",
       " 'big': 349,\n",
       " 'branch': 350,\n",
       " 'cut': 351,\n",
       " 'fiercely': 352,\n",
       " 'him': 353,\n",
       " 'seize': 354,\n",
       " 'behind': 355,\n",
       " 'replied': 356,\n",
       " 'rock': 357,\n",
       " 'seen': 358,\n",
       " 'stepped': 359,\n",
       " 'strangest': 360,\n",
       " 'travelers': 361,\n",
       " 'bend': 362,\n",
       " 'did': 363,\n",
       " 'forest': 364,\n",
       " 'given': 365,\n",
       " 'keep': 366,\n",
       " 'minds': 367,\n",
       " 'only': 368,\n",
       " 'order': 369,\n",
       " 'policemen': 370,\n",
       " 'power': 371,\n",
       " 'probably': 372,\n",
       " 'row': 373,\n",
       " 'strangers': 374,\n",
       " 'these': 375,\n",
       " 'wonderful': 376,\n",
       " 'dance': 377,\n",
       " 'join': 378,\n",
       " 'won': 379,\n",
       " 'books': 380,\n",
       " 'found': 381,\n",
       " 'funny': 382,\n",
       " 'laugh': 383,\n",
       " 'pictures': 384,\n",
       " 'laughed': 385,\n",
       " 'shore': 386,\n",
       " 'swam': 387,\n",
       " 'top': 388,\n",
       " 'water': 389,\n",
       " 'head': 390,\n",
       " 'sticking': 391,\n",
       " 'those': 392,\n",
       " 'why': 393,\n",
       " 'adventures': 394,\n",
       " 'companions': 395,\n",
       " 'cyclone': 396,\n",
       " 'met': 397,\n",
       " 'oz': 398,\n",
       " 'story': 399,\n",
       " 'told': 400,\n",
       " 'absurd': 401,\n",
       " 'bowed': 402,\n",
       " 'dare': 403,\n",
       " 'grave': 404,\n",
       " 'looking': 405,\n",
       " 'say': 406,\n",
       " 'simply': 407,\n",
       " 'solemn': 408,\n",
       " 'thimble': 409,\n",
       " 'thing': 410,\n",
       " 'whole': 411,\n",
       " 'right': 412,\n",
       " 'shoe': 413,\n",
       " 'wow': 414,\n",
       " 'world': 415,\n",
       " 'hush': 416,\n",
       " 'happened': 417,\n",
       " 'unknown': 418,\n",
       " 'beginning': 419,\n",
       " 'different': 420,\n",
       " 'going': 421,\n",
       " 'morning': 422,\n",
       " 'timidly': 423,\n",
       " 'use': 424,\n",
       " 'yesterday': 425,\n",
       " 'another': 426,\n",
       " 'upon': 427,\n",
       " 'anxious': 428,\n",
       " 'yes': 429,\n",
       " 'during': 430,\n",
       " 'greatest': 431,\n",
       " 'known': 432,\n",
       " 'loss': 433,\n",
       " 'terrible': 434,\n",
       " 'undergo': 435,\n",
       " 'year': 436,\n",
       " 'bottom': 437,\n",
       " 'country': 438,\n",
       " 'covered': 439,\n",
       " 'four': 440,\n",
       " 'gloom': 441,\n",
       " 'hill': 442,\n",
       " 'its': 443,\n",
       " 'passed': 444,\n",
       " 'pieces': 445,\n",
       " 'quadlings': 446,\n",
       " 'rest': 447,\n",
       " 'safety': 448,\n",
       " 'steep': 449,\n",
       " 'foot': 450,\n",
       " 'standing': 451,\n",
       " 'sneezing': 452,\n",
       " 'bed': 453,\n",
       " 'door': 454,\n",
       " 'heels': 455,\n",
       " 'opened': 456,\n",
       " 'ran': 457,\n",
       " 'sprang': 458,\n",
       " 'toto': 459,\n",
       " 'long': 460,\n",
       " 'sister': 461,\n",
       " 'sleep': 462,\n",
       " 'breath': 463,\n",
       " 'caterpillar': 464,\n",
       " 'curious': 465,\n",
       " 'drew': 466,\n",
       " 'father': 467,\n",
       " 'listeners': 468,\n",
       " 'mock': 469,\n",
       " 'perfectly': 470,\n",
       " 'quiet': 471,\n",
       " 'repeating': 472,\n",
       " 'turtle': 473,\n",
       " 'william': 474,\n",
       " 'calling': 475,\n",
       " 'east': 476,\n",
       " 'possibly': 477,\n",
       " 'saying': 478,\n",
       " 'sorceress': 479,\n",
       " 'woman': 480,\n",
       " 'who': 481,\n",
       " 'been': 482,\n",
       " 'clothes': 483,\n",
       " 'clown': 484,\n",
       " 'completely': 485,\n",
       " 'every': 486,\n",
       " 'indeed': 487,\n",
       " 'jolly': 488,\n",
       " 'many': 489,\n",
       " 'mended': 490,\n",
       " 'places': 491,\n",
       " 'plainly': 492,\n",
       " 'red': 493,\n",
       " 'showing': 494,\n",
       " 'spite': 495,\n",
       " 'toward': 496,\n",
       " 'walking': 497,\n",
       " 'yellow': 498,\n",
       " 'exactly': 499,\n",
       " 'giving': 500,\n",
       " 'himself': 501,\n",
       " 'left': 502,\n",
       " 'lion': 503,\n",
       " 'smiled': 504,\n",
       " 'success': 505,\n",
       " 'wanted': 506,\n",
       " 'just': 507,\n",
       " 'looks': 508,\n",
       " 'baby': 509,\n",
       " 'believed': 510,\n",
       " 'bells': 511,\n",
       " 'boy': 512,\n",
       " 'busy': 513,\n",
       " 'cattle': 514,\n",
       " 'change': 515,\n",
       " 'clamour': 516,\n",
       " 'closed': 517,\n",
       " 'confused': 518,\n",
       " 'cries': 519,\n",
       " 'distance': 520,\n",
       " 'dull': 521,\n",
       " 'eyes': 522,\n",
       " 'farm': 523,\n",
       " 'half': 524,\n",
       " 'heavy': 525,\n",
       " 'knew': 526,\n",
       " 'lowing': 527,\n",
       " 'noises': 528,\n",
       " 'place': 529,\n",
       " 'pool': 530,\n",
       " 'rattling': 531,\n",
       " 'reality': 532,\n",
       " 'reeds': 533,\n",
       " 'rippling': 534,\n",
       " 'rustling': 535,\n",
       " 'sheep': 536,\n",
       " 'shepherd': 537,\n",
       " 'shriek': 538,\n",
       " 'shrill': 539,\n",
       " 'sneeze': 540,\n",
       " 'sobs': 541,\n",
       " 'teacups': 542,\n",
       " 'though': 543,\n",
       " 'thy': 544,\n",
       " 'tinkling': 545,\n",
       " 'waving': 546,\n",
       " 'while': 547,\n",
       " 'wonderland': 548,\n",
       " 'retorted': 549,\n",
       " 'tried': 550,\n",
       " 'far': 551,\n",
       " 'pleased': 552,\n",
       " 'bread': 553,\n",
       " 'crumbs': 554,\n",
       " 'grumbled': 555,\n",
       " 'hatter': 556,\n",
       " 'knife': 557,\n",
       " 'put': 558,\n",
       " 'shouldn': 559,\n",
       " 'asleep': 560,\n",
       " 'fast': 561,\n",
       " 'fell': 562,\n",
       " 'forgot': 563,\n",
       " 'poppies': 564,\n",
       " 'where': 565,\n",
       " 'feeling': 566,\n",
       " 'ann': 567,\n",
       " 'doing': 568,\n",
       " 'hunting': 569,\n",
       " 'mary': 570,\n",
       " 'noticed': 571,\n",
       " 'tone': 572,\n",
       " 'we': 573,\n",
       " 'axe': 574,\n",
       " 'friend': 575,\n",
       " 'carried': 576,\n",
       " 'everything': 577,\n",
       " 'balloon': 578,\n",
       " 'matters': 579,\n",
       " 'talked': 580,\n",
       " 'growl': 581,\n",
       " 'tail': 582,\n",
       " 'wag': 583,\n",
       " 'help': 584,\n",
       " 'believe': 585,\n",
       " 'making': 586,\n",
       " 'hold': 587,\n",
       " 'tongue': 588,\n",
       " 'friendly': 589,\n",
       " 'munchkins': 590,\n",
       " 'pass': 591,\n",
       " 'people': 592,\n",
       " 'seem': 593,\n",
       " 'unable': 594,\n",
       " 'gates': 595,\n",
       " 'guardian': 596,\n",
       " 'kalidahs': 597,\n",
       " 'terribly': 598,\n",
       " 'jurors': 599,\n",
       " 'shoulders': 600,\n",
       " 'stupid': 601,\n",
       " 'writing': 602,\n",
       " 'although': 603,\n",
       " 'city': 604,\n",
       " 'emerald': 605,\n",
       " 'ruler': 606,\n",
       " 'box': 607,\n",
       " 'court': 608,\n",
       " 'even': 609,\n",
       " 'guessed': 610,\n",
       " 'hand': 611,\n",
       " 'near': 612,\n",
       " 'pepper': 613,\n",
       " 'beautiful': 614,\n",
       " 'has': 615,\n",
       " 'living': 616,\n",
       " 'permitted': 617,\n",
       " 'times': 618,\n",
       " 'became': 619,\n",
       " 'care': 620,\n",
       " 'died': 621,\n",
       " 'grew': 622,\n",
       " 'lived': 623,\n",
       " 'mother': 624,\n",
       " 'woodchopper': 625,\n",
       " 'beau': 626,\n",
       " 'oop': 627,\n",
       " 'ootiful': 628,\n",
       " 'soo': 629,\n",
       " 'dish': 630,\n",
       " 'heads': 631,\n",
       " 'higher': 632,\n",
       " 'smooth': 633,\n",
       " 'surface': 634,\n",
       " 'than': 635,\n",
       " 'courage': 636,\n",
       " 'cry': 637,\n",
       " 'fright': 638,\n",
       " 'gave': 639,\n",
       " 'also': 640,\n",
       " 'having': 641,\n",
       " 'everyone': 642,\n",
       " 'fooled': 643,\n",
       " 'argument': 644,\n",
       " 'done': 645,\n",
       " 'everybody': 646,\n",
       " 'executed': 647,\n",
       " 'less': 648,\n",
       " 'round': 649,\n",
       " 'wasn': 650,\n",
       " 'cat': 651,\n",
       " 'enough': 652,\n",
       " 're': 653,\n",
       " 'walk': 654,\n",
       " 'brother': 655,\n",
       " 'grammar': 656,\n",
       " 'latin': 657,\n",
       " 'mouse': 658,\n",
       " 'remembered': 659,\n",
       " 'speaking': 660,\n",
       " 'added': 661,\n",
       " 'hear': 662,\n",
       " 'let': 663,\n",
       " 'beyond': 664,\n",
       " 'powers': 665,\n",
       " 'quite': 666,\n",
       " 'thinking': 667,\n",
       " 'aunt': 668,\n",
       " 'em': 669,\n",
       " 'mantel': 670,\n",
       " 'stand': 671,\n",
       " 'explain': 672,\n",
       " 'yourself': 673,\n",
       " 'carrier': 674,\n",
       " 'feet': 675,\n",
       " 'own': 676,\n",
       " 'presents': 677,\n",
       " 'sending': 678,\n",
       " 'delightful': 679,\n",
       " 'hate': 680,\n",
       " 'history': 681,\n",
       " 'offended': 682,\n",
       " 'promised': 683,\n",
       " 'whisper': 684,\n",
       " 'certainly': 685,\n",
       " 'constant': 686,\n",
       " 'crash': 687,\n",
       " 'extraordinary': 688,\n",
       " 'howling': 689,\n",
       " 'kettle': 690,\n",
       " 'most': 691,\n",
       " 'noise': 692,\n",
       " 'within': 693,\n",
       " 'command': 694,\n",
       " 'still': 695,\n",
       " 'hookah': 696,\n",
       " 'itself': 697,\n",
       " 'minute': 698,\n",
       " 'mouth': 699,\n",
       " 'shook': 700,\n",
       " 'twice': 701,\n",
       " 'yawned': 702,\n",
       " 'curiously': 703,\n",
       " 'entered': 704,\n",
       " 'face': 705,\n",
       " 'look': 706,\n",
       " 'whispered': 707,\n",
       " 'eagerly': 708,\n",
       " 'executioner': 709,\n",
       " 'fetch': 710,\n",
       " 'king': 711,\n",
       " 'myself': 712,\n",
       " 'cook': 713,\n",
       " 'joined': 714,\n",
       " 'accident': 715,\n",
       " 'diligently': 716,\n",
       " 'gazing': 717,\n",
       " 'handed': 718,\n",
       " 'jury': 719,\n",
       " 'lizard': 720,\n",
       " 'overcome': 721,\n",
       " 'pencils': 722,\n",
       " 'recovered': 723,\n",
       " 'set': 724,\n",
       " 'shock': 725,\n",
       " 'sit': 726,\n",
       " 'slates': 727,\n",
       " 'upset': 728,\n",
       " 'write': 729,\n",
       " 'call': 730,\n",
       " 'carefully': 731,\n",
       " 'dry': 732,\n",
       " 'leave': 733,\n",
       " 'listened': 734,\n",
       " 'understand': 735,\n",
       " 'wish': 736,\n",
       " 'dreadfully': 737,\n",
       " 'handsome': 738,\n",
       " 'pig': 739,\n",
       " 'ugly': 740,\n",
       " 'cross': 741,\n",
       " 'mouths': 742,\n",
       " 'tails': 743,\n",
       " 'mind': 744,\n",
       " 'uncomfortable': 745,\n",
       " 'ah': 746,\n",
       " 'frightened': 747,\n",
       " 'motions': 748,\n",
       " 'relieve': 749,\n",
       " 'boots': 750,\n",
       " 'deep': 751,\n",
       " 'sea': 752,\n",
       " 'under': 753,\n",
       " 'whiting': 754,\n",
       " 'shouted': 755,\n",
       " 'anybody': 756,\n",
       " 'eating': 757,\n",
       " 'flying': 758,\n",
       " 'free': 759,\n",
       " 'fruit': 760,\n",
       " 'happily': 761,\n",
       " 'master': 762,\n",
       " 'nuts': 763,\n",
       " 'tree': 764,\n",
       " 'without': 765,\n",
       " 'center': 766,\n",
       " 'exact': 767,\n",
       " 'house': 768,\n",
       " 'north': 769,\n",
       " 'south': 770,\n",
       " 'winds': 771,\n",
       " 'doorway': 772,\n",
       " 'politeness': 773,\n",
       " 'spot': 774,\n",
       " 'stupidly': 775,\n",
       " 'wait': 776,\n",
       " 'wasted': 777,\n",
       " 'groaning': 778,\n",
       " 'clear': 779,\n",
       " 'cool': 780,\n",
       " 'drink': 781,\n",
       " 'emeralds': 782,\n",
       " 'helped': 783,\n",
       " 'journey': 784,\n",
       " 'ready': 785,\n",
       " 'barrowful': 786,\n",
       " 'may': 787,\n",
       " 'arms': 788,\n",
       " 'forehead': 789,\n",
       " 'grinning': 790,\n",
       " 'hairy': 791,\n",
       " 'kiss': 792,\n",
       " 'mark': 793,\n",
       " 'motioning': 794,\n",
       " 'others': 795,\n",
       " 'short': 796,\n",
       " 'stopped': 797,\n",
       " 'stretched': 798,\n",
       " 'touch': 799,\n",
       " 'gloves': 800,\n",
       " 'hands': 801,\n",
       " 'kid': 802,\n",
       " 'talking': 803,\n",
       " 'white': 804,\n",
       " 'cap': 805,\n",
       " 'kingdom': 806,\n",
       " 'present': 807,\n",
       " 'princess': 808,\n",
       " 'quelala': 809,\n",
       " 'wedding': 810,\n",
       " 'lesson': 811,\n",
       " 'serve': 812,\n",
       " 'step': 813,\n",
       " 'farmer': 814,\n",
       " 'munchkin': 815,\n",
       " 'certain': 816,\n",
       " 'fly': 817,\n",
       " 'along': 818,\n",
       " 'de': 819,\n",
       " 'friends': 820,\n",
       " 'happy': 821,\n",
       " 'hugged': 822,\n",
       " 'ri': 823,\n",
       " 'sang': 824,\n",
       " 'tol': 825,\n",
       " 'float': 826,\n",
       " 'remarked': 827,\n",
       " 'bottle': 828,\n",
       " 'fan': 829,\n",
       " 'hoped': 830,\n",
       " 'pair': 831,\n",
       " 'pairs': 832,\n",
       " 'table': 833,\n",
       " 'tidy': 834,\n",
       " 'tiny': 835,\n",
       " 'question': 836,\n",
       " 'darkness': 837,\n",
       " 'faded': 838,\n",
       " 'hour': 839,\n",
       " 'light': 840,\n",
       " 'stumbling': 841,\n",
       " 'themselves': 842,\n",
       " 'butter': 843,\n",
       " 'piece': 844,\n",
       " 'teacup': 845,\n",
       " 'arts': 846,\n",
       " 'bar': 847,\n",
       " 'floor': 848,\n",
       " 'human': 849,\n",
       " 'invisible': 850,\n",
       " 'iron': 851,\n",
       " 'kitchen': 852,\n",
       " 'middle': 853,\n",
       " 'placed': 854,\n",
       " 'smoking': 855,\n",
       " 'calmly': 856,\n",
       " 'school': 857,\n",
       " 'sobbing': 858,\n",
       " 'ears': 859,\n",
       " 'groan': 860,\n",
       " 'reached': 861,\n",
       " 'sound': 862,\n",
       " 'continued': 863,\n",
       " 'highest': 864,\n",
       " 'needs': 865,\n",
       " 'pigeon': 866,\n",
       " 'raising': 867,\n",
       " 'sky': 868,\n",
       " 'wood': 869,\n",
       " 'wriggling': 870,\n",
       " 'appeared': 871,\n",
       " 'afford': 872,\n",
       " 'better': 873,\n",
       " 'crops': 874,\n",
       " 'dreadful': 875,\n",
       " 'henry': 876,\n",
       " 'mourning': 877,\n",
       " 'surely': 878,\n",
       " 'uncle': 879,\n",
       " 'deadly': 880,\n",
       " 'flower': 881,\n",
       " 'run': 882,\n",
       " 'pile': 883,\n",
       " 'shaggy': 884,\n",
       " 'wolves': 885,\n",
       " 'interested': 886,\n",
       " 'marble': 887,\n",
       " 'joints': 888,\n",
       " 'oil': 889,\n",
       " 'growing': 890,\n",
       " 'saved': 891,\n",
       " 'surroundings': 892,\n",
       " 'hours': 893,\n",
       " 'lessons': 894,\n",
       " 'bucket': 895,\n",
       " 'melted': 896,\n",
       " 'mess': 897,\n",
       " 'seeing': 898,\n",
       " 'threw': 899,\n",
       " 'belonged': 900,\n",
       " 'charm': 901,\n",
       " 'fear': 902,\n",
       " 'tremble': 903,\n",
       " 'bringing': 904,\n",
       " 'finished': 905,\n",
       " 'hadn': 906,\n",
       " 'majesty': 907,\n",
       " 'tea': 908,\n",
       " 'favor': 909,\n",
       " 'grateful': 910,\n",
       " 'draw': 911,\n",
       " 'eh': 912,\n",
       " 'treacle': 913,\n",
       " 'asking': 914,\n",
       " 'crowded': 915,\n",
       " 'panting': 916,\n",
       " 'pray': 917,\n",
       " 'speak': 918,\n",
       " 'don': 919,\n",
       " 'longer': 920,\n",
       " 'trouble': 921,\n",
       " 'finish': 922,\n",
       " 'list': 923,\n",
       " 'reading': 924,\n",
       " 'singers': 925,\n",
       " 'talk': 926,\n",
       " 'chin': 927,\n",
       " 'height': 928,\n",
       " 'keeping': 929,\n",
       " 'secondly': 930,\n",
       " 'sharp': 931,\n",
       " 'shoulder': 932,\n",
       " 'uncomfortably': 933,\n",
       " 'either': 934,\n",
       " 'key': 935,\n",
       " 'large': 936,\n",
       " 'locks': 937,\n",
       " 'rate': 938,\n",
       " 'small': 939,\n",
       " 'always': 940,\n",
       " 'between': 941,\n",
       " 'sigh': 942,\n",
       " 'wash': 943,\n",
       " 'whiles': 944,\n",
       " 'confidence': 945,\n",
       " 'need': 946,\n",
       " 'beast': 947,\n",
       " 'fallen': 948,\n",
       " 'fields': 949,\n",
       " 'flowers': 950,\n",
       " 'huge': 951,\n",
       " 'poppy': 952,\n",
       " 'spread': 953,\n",
       " 'strong': 954,\n",
       " 'sweet': 955,\n",
       " 'become': 956,\n",
       " 'dominions': 957,\n",
       " 'instead': 958,\n",
       " 'praying': 959,\n",
       " 'therefore': 960,\n",
       " 'destroying': 961,\n",
       " 'news': 962,\n",
       " 'reply': 963,\n",
       " 'soldier': 964,\n",
       " 'straight': 965,\n",
       " 'bright': 966,\n",
       " 'manner': 967,\n",
       " 'sparkle': 968,\n",
       " 'allow': 969,\n",
       " 'cowardly': 970,\n",
       " 'easily': 971,\n",
       " 'scare': 972,\n",
       " 'seems': 973,\n",
       " 'creep': 974,\n",
       " 'eat': 975,\n",
       " 'garden': 976,\n",
       " 'happens': 977,\n",
       " 'larger': 978,\n",
       " 'reach': 979,\n",
       " 'smaller': 980,\n",
       " 'felt': 981,\n",
       " 'gravely': 982,\n",
       " 'irritated': 983,\n",
       " 'ought': 984,\n",
       " 'remarks': 985,\n",
       " 'begin': 986,\n",
       " 'stop': 987,\n",
       " 'best': 988,\n",
       " 'dangers': 989,\n",
       " 'glinda': 990,\n",
       " 'travel': 991,\n",
       " 'blow': 992,\n",
       " 'claws': 993,\n",
       " 'edge': 994,\n",
       " 'paw': 995,\n",
       " 'road': 996,\n",
       " 'spinning': 997,\n",
       " 'struck': 998,\n",
       " 'chorus': 999,\n",
       " ...}"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dictionary.token2id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "B2R6GIb0PIo5",
    "outputId": "084c8d09-ff4b-4989-d033-16cf4a1a3572"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['all', 'and', 'at', 'away', 'began']"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocab = list(dictionary.token2id.keys())\n",
    "vocab[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "MaZdrWZDPIo5",
    "outputId": "2a945f83-17b7-44ec-8b83-09c46b975c16"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{15: 369,\n",
       " 17: 3718,\n",
       " 23: 138,\n",
       " 21: 102,\n",
       " 2: 374,\n",
       " 13: 79,\n",
       " 18: 1470,\n",
       " 19: 32,\n",
       " 1: 2026,\n",
       " 4: 70,\n",
       " 24: 24,\n",
       " 8: 460,\n",
       " 16: 49,\n",
       " 11: 101,\n",
       " 20: 9,\n",
       " 14: 189,\n",
       " 12: 1099,\n",
       " 10: 1,\n",
       " 7: 134,\n",
       " 22: 95,\n",
       " 6: 7,\n",
       " 3: 72,\n",
       " 0: 339,\n",
       " 9: 10,\n",
       " 5: 11,\n",
       " 25: 29,\n",
       " 33: 717,\n",
       " 30: 7,\n",
       " 29: 21,\n",
       " 28: 32,\n",
       " 26: 194,\n",
       " 27: 10,\n",
       " 31: 63,\n",
       " 32: 109,\n",
       " 35: 110,\n",
       " 36: 80,\n",
       " 34: 19,\n",
       " 41: 165,\n",
       " 37: 149,\n",
       " 43: 8,\n",
       " 42: 613,\n",
       " 38: 208,\n",
       " 39: 63,\n",
       " 40: 5,\n",
       " 44: 310,\n",
       " 46: 4,\n",
       " 45: 28,\n",
       " 59: 765,\n",
       " 62: 688,\n",
       " 55: 4,\n",
       " 61: 267,\n",
       " 47: 120,\n",
       " 49: 467,\n",
       " 60: 24,\n",
       " 50: 377,\n",
       " 56: 1,\n",
       " 57: 44,\n",
       " 53: 6,\n",
       " 52: 12,\n",
       " 58: 4,\n",
       " 48: 22,\n",
       " 51: 2,\n",
       " 54: 99,\n",
       " 66: 360,\n",
       " 65: 11,\n",
       " 67: 522,\n",
       " 64: 23,\n",
       " 68: 10,\n",
       " 63: 119,\n",
       " 69: 217,\n",
       " 70: 94,\n",
       " 78: 73,\n",
       " 79: 30,\n",
       " 75: 99,\n",
       " 72: 115,\n",
       " 77: 36,\n",
       " 80: 362,\n",
       " 73: 255,\n",
       " 76: 675,\n",
       " 74: 3,\n",
       " 71: 140,\n",
       " 83: 184,\n",
       " 82: 86,\n",
       " 85: 5,\n",
       " 84: 156,\n",
       " 81: 9,\n",
       " 87: 99,\n",
       " 86: 73,\n",
       " 99: 66,\n",
       " 93: 3,\n",
       " 105: 11,\n",
       " 89: 3,\n",
       " 100: 82,\n",
       " 88: 1,\n",
       " 109: 249,\n",
       " 102: 3,\n",
       " 104: 2,\n",
       " 94: 26,\n",
       " 98: 33,\n",
       " 108: 457,\n",
       " 103: 58,\n",
       " 95: 10,\n",
       " 92: 9,\n",
       " 97: 2,\n",
       " 101: 3,\n",
       " 106: 569,\n",
       " 91: 1,\n",
       " 110: 160,\n",
       " 90: 2,\n",
       " 107: 125,\n",
       " 96: 8,\n",
       " 112: 39,\n",
       " 113: 23,\n",
       " 117: 184,\n",
       " 114: 6,\n",
       " 119: 123,\n",
       " 123: 28,\n",
       " 120: 10,\n",
       " 111: 79,\n",
       " 115: 14,\n",
       " 121: 19,\n",
       " 122: 196,\n",
       " 116: 81,\n",
       " 118: 64,\n",
       " 128: 122,\n",
       " 125: 294,\n",
       " 127: 105,\n",
       " 124: 89,\n",
       " 126: 106,\n",
       " 130: 20,\n",
       " 135: 7,\n",
       " 134: 27,\n",
       " 131: 89,\n",
       " 136: 44,\n",
       " 133: 91,\n",
       " 132: 20,\n",
       " 129: 1,\n",
       " 142: 19,\n",
       " 139: 12,\n",
       " 143: 24,\n",
       " 140: 9,\n",
       " 141: 4,\n",
       " 137: 159,\n",
       " 138: 7,\n",
       " 150: 173,\n",
       " 145: 9,\n",
       " 152: 78,\n",
       " 146: 23,\n",
       " 149: 4,\n",
       " 148: 2,\n",
       " 151: 6,\n",
       " 147: 256,\n",
       " 144: 47,\n",
       " 155: 13,\n",
       " 158: 794,\n",
       " 157: 116,\n",
       " 163: 28,\n",
       " 159: 9,\n",
       " 167: 23,\n",
       " 153: 6,\n",
       " 166: 21,\n",
       " 160: 31,\n",
       " 156: 19,\n",
       " 164: 52,\n",
       " 161: 14,\n",
       " 165: 9,\n",
       " 162: 36,\n",
       " 154: 20,\n",
       " 177: 46,\n",
       " 183: 5,\n",
       " 175: 14,\n",
       " 180: 26,\n",
       " 182: 102,\n",
       " 179: 1,\n",
       " 174: 54,\n",
       " 170: 2,\n",
       " 168: 75,\n",
       " 172: 2,\n",
       " 178: 279,\n",
       " 173: 37,\n",
       " 176: 17,\n",
       " 181: 183,\n",
       " 169: 2,\n",
       " 171: 2,\n",
       " 190: 19,\n",
       " 186: 21,\n",
       " 185: 9,\n",
       " 195: 104,\n",
       " 194: 1,\n",
       " 192: 10,\n",
       " 184: 92,\n",
       " 191: 39,\n",
       " 188: 157,\n",
       " 193: 65,\n",
       " 187: 160,\n",
       " 189: 53,\n",
       " 198: 173,\n",
       " 196: 36,\n",
       " 197: 84,\n",
       " 206: 19,\n",
       " 199: 2,\n",
       " 204: 1,\n",
       " 202: 35,\n",
       " 201: 14,\n",
       " 203: 3,\n",
       " 200: 2,\n",
       " 205: 22,\n",
       " 208: 51,\n",
       " 207: 11,\n",
       " 210: 17,\n",
       " 209: 35,\n",
       " 214: 42,\n",
       " 211: 39,\n",
       " 213: 23,\n",
       " 212: 128,\n",
       " 217: 381,\n",
       " 219: 191,\n",
       " 220: 13,\n",
       " 218: 57,\n",
       " 215: 31,\n",
       " 216: 2,\n",
       " 221: 4,\n",
       " 222: 68,\n",
       " 223: 1,\n",
       " 224: 1,\n",
       " 233: 41,\n",
       " 229: 8,\n",
       " 230: 24,\n",
       " 226: 9,\n",
       " 225: 26,\n",
       " 231: 304,\n",
       " 234: 105,\n",
       " 232: 21,\n",
       " 228: 22,\n",
       " 235: 10,\n",
       " 227: 2,\n",
       " 237: 3,\n",
       " 236: 51,\n",
       " 238: 15,\n",
       " 244: 180,\n",
       " 243: 12,\n",
       " 240: 35,\n",
       " 241: 2,\n",
       " 242: 3,\n",
       " 239: 24,\n",
       " 256: 48,\n",
       " 251: 212,\n",
       " 248: 81,\n",
       " 245: 39,\n",
       " 253: 8,\n",
       " 250: 59,\n",
       " 258: 165,\n",
       " 255: 8,\n",
       " 249: 67,\n",
       " 257: 25,\n",
       " 254: 7,\n",
       " 247: 14,\n",
       " 252: 70,\n",
       " 246: 32,\n",
       " 259: 4,\n",
       " 260: 51,\n",
       " 262: 18,\n",
       " 261: 30,\n",
       " 265: 64,\n",
       " 264: 2,\n",
       " 263: 55,\n",
       " 266: 1,\n",
       " 269: 7,\n",
       " 267: 8,\n",
       " 268: 43,\n",
       " 273: 293,\n",
       " 271: 119,\n",
       " 276: 1,\n",
       " 274: 6,\n",
       " 272: 99,\n",
       " 270: 19,\n",
       " 275: 4,\n",
       " 277: 21,\n",
       " 281: 17,\n",
       " 279: 17,\n",
       " 280: 37,\n",
       " 278: 1,\n",
       " 283: 15,\n",
       " 286: 3,\n",
       " 285: 8,\n",
       " 291: 100,\n",
       " 289: 3,\n",
       " 293: 17,\n",
       " 284: 17,\n",
       " 290: 14,\n",
       " 282: 4,\n",
       " 287: 18,\n",
       " 292: 36,\n",
       " 288: 19,\n",
       " 294: 28,\n",
       " 297: 179,\n",
       " 296: 146,\n",
       " 295: 13,\n",
       " 302: 94,\n",
       " 304: 69,\n",
       " 299: 2,\n",
       " 303: 6,\n",
       " 305: 45,\n",
       " 301: 2,\n",
       " 298: 11,\n",
       " 300: 44,\n",
       " 306: 64,\n",
       " 308: 3,\n",
       " 309: 35,\n",
       " 307: 8,\n",
       " 311: 11,\n",
       " 310: 13,\n",
       " 312: 1,\n",
       " 314: 7,\n",
       " 313: 54,\n",
       " 319: 49,\n",
       " 320: 93,\n",
       " 321: 49,\n",
       " 316: 4,\n",
       " 322: 21,\n",
       " 318: 43,\n",
       " 323: 94,\n",
       " 315: 24,\n",
       " 317: 13,\n",
       " 325: 115,\n",
       " 324: 14,\n",
       " 328: 69,\n",
       " 327: 38,\n",
       " 326: 2,\n",
       " 329: 4,\n",
       " 330: 2,\n",
       " 331: 30,\n",
       " 333: 112,\n",
       " 332: 5,\n",
       " 337: 3,\n",
       " 336: 74,\n",
       " 334: 11,\n",
       " 335: 10,\n",
       " 338: 9,\n",
       " 339: 6,\n",
       " 341: 53,\n",
       " 345: 6,\n",
       " 340: 2,\n",
       " 342: 20,\n",
       " 344: 67,\n",
       " 343: 12,\n",
       " 347: 1,\n",
       " 346: 1,\n",
       " 349: 44,\n",
       " 350: 4,\n",
       " 348: 5,\n",
       " 354: 1,\n",
       " 353: 181,\n",
       " 352: 3,\n",
       " 351: 18,\n",
       " 356: 62,\n",
       " 359: 5,\n",
       " 355: 24,\n",
       " 357: 5,\n",
       " 360: 2,\n",
       " 361: 16,\n",
       " 358: 24,\n",
       " 364: 38,\n",
       " 363: 114,\n",
       " 366: 25,\n",
       " 367: 2,\n",
       " 368: 80,\n",
       " 373: 4,\n",
       " 362: 4,\n",
       " 372: 3,\n",
       " 375: 43,\n",
       " 370: 1,\n",
       " 365: 6,\n",
       " 376: 18,\n",
       " 371: 13,\n",
       " 369: 6,\n",
       " 374: 10,\n",
       " 379: 26,\n",
       " 378: 7,\n",
       " 377: 12,\n",
       " 380: 3,\n",
       " 381: 61,\n",
       " 384: 5,\n",
       " 383: 7,\n",
       " 382: 7,\n",
       " 385: 10,\n",
       " 388: 20,\n",
       " 389: 25,\n",
       " 387: 6,\n",
       " 386: 7,\n",
       " 393: 66,\n",
       " 392: 25,\n",
       " 391: 4,\n",
       " 390: 113,\n",
       " 400: 28,\n",
       " 399: 14,\n",
       " 396: 9,\n",
       " 398: 124,\n",
       " 395: 7,\n",
       " 394: 7,\n",
       " 397: 12,\n",
       " 411: 13,\n",
       " 410: 59,\n",
       " 401: 2,\n",
       " 404: 3,\n",
       " 403: 10,\n",
       " 406: 55,\n",
       " 407: 6,\n",
       " 402: 11,\n",
       " 409: 2,\n",
       " 405: 38,\n",
       " 408: 5,\n",
       " 412: 40,\n",
       " 413: 4,\n",
       " 414: 5,\n",
       " 415: 19,\n",
       " 416: 3,\n",
       " 417: 13,\n",
       " 418: 1,\n",
       " 419: 13,\n",
       " 422: 26,\n",
       " 423: 9,\n",
       " 424: 26,\n",
       " 421: 44,\n",
       " 425: 4,\n",
       " 420: 9,\n",
       " 426: 38,\n",
       " 427: 85,\n",
       " 429: 26,\n",
       " 428: 8,\n",
       " 434: 21,\n",
       " 435: 2,\n",
       " 430: 5,\n",
       " 436: 5,\n",
       " 431: 6,\n",
       " 433: 3,\n",
       " 432: 6,\n",
       " 438: 58,\n",
       " 446: 9,\n",
       " 440: 20,\n",
       " 444: 18,\n",
       " 447: 24,\n",
       " 448: 1,\n",
       " 443: 72,\n",
       " 441: 1,\n",
       " 449: 2,\n",
       " 442: 10,\n",
       " 439: 10,\n",
       " 437: 11,\n",
       " 445: 13,\n",
       " 451: 12,\n",
       " 450: 15,\n",
       " 452: 5,\n",
       " 458: 3,\n",
       " 453: 18,\n",
       " 459: 68,\n",
       " 455: 3,\n",
       " 457: 28,\n",
       " 456: 19,\n",
       " 454: 44,\n",
       " 461: 8,\n",
       " 460: 75,\n",
       " 462: 17,\n",
       " 468: 1,\n",
       " 470: 7,\n",
       " 471: 4,\n",
       " 472: 3,\n",
       " 467: 6,\n",
       " 474: 4,\n",
       " 464: 23,\n",
       " 469: 47,\n",
       " 473: 50,\n",
       " 466: 7,\n",
       " 463: 5,\n",
       " 465: 19,\n",
       " 480: 30,\n",
       " 477: 4,\n",
       " 475: 3,\n",
       " 479: 3,\n",
       " 478: 15,\n",
       " 476: 15,\n",
       " 481: 144,\n",
       " 487: 29,\n",
       " 488: 1,\n",
       " 484: 3,\n",
       " 497: 17,\n",
       " 496: 17,\n",
       " 495: 4,\n",
       " 483: 10,\n",
       " 493: 11,\n",
       " 498: 27,\n",
       " 485: 2,\n",
       " 486: 31,\n",
       " 494: 3,\n",
       " 492: 2,\n",
       " 482: 82,\n",
       " 490: 4,\n",
       " 489: 51,\n",
       " 491: 4,\n",
       " 502: 34,\n",
       " 501: 29,\n",
       " 504: 4,\n",
       " 505: 1,\n",
       " 500: 3,\n",
       " 503: 136,\n",
       " 499: 10,\n",
       " 506: 10,\n",
       " 508: 3,\n",
       " 507: 73,\n",
       " 517: 5,\n",
       " 522: 68,\n",
       " 524: 22,\n",
       " 510: 1,\n",
       " 548: 2,\n",
       " 543: 10,\n",
       " 526: 34,\n",
       " 515: 11,\n",
       " 521: 3,\n",
       " 532: 2,\n",
       " 535: 1,\n",
       " 530: 10,\n",
       " 534: 2,\n",
       " 546: 4,\n",
       " 533: 1,\n",
       " 531: 3,\n",
       " 542: 2,\n",
       " 545: 2,\n",
       " 536: 3,\n",
       " 511: 2,\n",
       " 539: 6,\n",
       " 519: 1,\n",
       " 537: 1,\n",
       " 512: 4,\n",
       " 540: 2,\n",
       " 509: 15,\n",
       " 538: 5,\n",
       " 544: 1,\n",
       " 528: 2,\n",
       " 518: 4,\n",
       " 516: 1,\n",
       " 513: 4,\n",
       " 523: 1,\n",
       " 547: 50,\n",
       " 527: 1,\n",
       " 514: 1,\n",
       " 520: 12,\n",
       " 529: 22,\n",
       " 525: 9,\n",
       " 541: 3,\n",
       " 550: 21,\n",
       " 549: 2,\n",
       " 552: 16,\n",
       " 551: 31,\n",
       " 554: 4,\n",
       " 556: 51,\n",
       " 555: 1,\n",
       " 559: 5,\n",
       " 558: 53,\n",
       " 553: 10,\n",
       " 557: 3,\n",
       " 563: 4,\n",
       " 565: 82,\n",
       " 562: 27,\n",
       " 564: 6,\n",
       " 561: 22,\n",
       " 560: 20,\n",
       " 566: 10,\n",
       " 571: 8,\n",
       " 569: 3,\n",
       " 572: 30,\n",
       " 570: 4,\n",
       " 567: 4,\n",
       " 568: 4,\n",
       " 573: 141,\n",
       " 574: 23,\n",
       " 575: 13,\n",
       " 577: 26,\n",
       " 576: 29,\n",
       " 578: 19,\n",
       " 580: 2,\n",
       " 579: 3,\n",
       " 581: 7,\n",
       " 583: 1,\n",
       " 582: 17,\n",
       " 584: 34,\n",
       " 586: 12,\n",
       " 585: 16,\n",
       " 587: 16,\n",
       " 588: 5,\n",
       " 592: 53,\n",
       " 593: 8,\n",
       " 589: 4,\n",
       " 590: 17,\n",
       " 594: 3,\n",
       " 591: 8,\n",
       " 596: 12,\n",
       " 595: 13,\n",
       " 598: 3,\n",
       " 597: 6,\n",
       " 600: 7,\n",
       " 599: 4,\n",
       " 602: 6,\n",
       " 601: 7,\n",
       " 606: 6,\n",
       " 605: 51,\n",
       " 604: 68,\n",
       " 603: 11,\n",
       " 613: 6,\n",
       " 607: 14,\n",
       " 611: 23,\n",
       " 610: 3,\n",
       " 609: 35,\n",
       " 608: 15,\n",
       " 612: 29,\n",
       " 618: 22,\n",
       " 614: 41,\n",
       " 617: 2,\n",
       " 616: 8,\n",
       " 615: 35,\n",
       " 622: 10,\n",
       " 619: 19,\n",
       " 625: 1,\n",
       " 621: 4,\n",
       " 620: 10,\n",
       " 624: 1,\n",
       " 623: 18,\n",
       " 626: 4,\n",
       " 628: 4,\n",
       " 629: 6,\n",
       " 627: 6,\n",
       " 633: 4,\n",
       " 634: 1,\n",
       " 630: 4,\n",
       " 632: 1,\n",
       " 635: 62,\n",
       " 631: 24,\n",
       " 636: 27,\n",
       " 639: 29,\n",
       " 637: 12,\n",
       " 638: 3,\n",
       " 640: 26,\n",
       " 641: 19,\n",
       " 643: 1,\n",
       " 642: 9,\n",
       " 644: 3,\n",
       " 650: 12,\n",
       " 645: 25,\n",
       " 648: 4,\n",
       " 646: 7,\n",
       " 647: 6,\n",
       " 649: 36,\n",
       " 653: 30,\n",
       " 651: 30,\n",
       " 654: 12,\n",
       " 652: 25,\n",
       " 660: 5,\n",
       " 658: 42,\n",
       " 659: 7,\n",
       " 655: 2,\n",
       " 657: 1,\n",
       " 656: 1,\n",
       " 661: 27,\n",
       " 663: 40,\n",
       " 662: 19,\n",
       " 666: 76,\n",
       " 664: 3,\n",
       " 665: 6,\n",
       " 667: 18,\n",
       " 671: 9,\n",
       " 668: 22,\n",
       " 669: 22,\n",
       " 670: 1,\n",
       " 672: 7,\n",
       " 673: 16,\n",
       " 674: 1,\n",
       " 678: 3,\n",
       " 677: 2,\n",
       " 676: 19,\n",
       " 675: 36,\n",
       " 679: 4,\n",
       " 683: 9,\n",
       " 681: 8,\n",
       " 680: 2,\n",
       " 684: 4,\n",
       " 682: 7,\n",
       " 685: 24,\n",
       " 691: 23,\n",
       " 688: 2,\n",
       " 692: 5,\n",
       " 693: 7,\n",
       " 686: 2,\n",
       " 689: 3,\n",
       " 687: 5,\n",
       " 690: 1,\n",
       " 695: 27,\n",
       " 694: 8,\n",
       " 698: 23,\n",
       " 696: 5,\n",
       " 699: 19,\n",
       " 702: 3,\n",
       " 701: 5,\n",
       " 700: 15,\n",
       " 697: 13,\n",
       " 704: 7,\n",
       " 703: 2,\n",
       " 707: 8,\n",
       " 706: 39,\n",
       " 705: 35,\n",
       " 710: 7,\n",
       " 709: 4,\n",
       " 712: 18,\n",
       " 711: 70,\n",
       " 708: 11,\n",
       " 713: 12,\n",
       " 714: 6,\n",
       " 719: 15,\n",
       " 723: 2,\n",
       " 725: 1,\n",
       " 728: 4,\n",
       " 727: 8,\n",
       " 722: 1,\n",
       " 718: 4,\n",
       " 724: 33,\n",
       " 716: 1,\n",
       " 729: 6,\n",
       " 715: 3,\n",
       " 720: 6,\n",
       " 721: 1,\n",
       " 726: 11,\n",
       " 717: 2,\n",
       " 734: 4,\n",
       " 731: 17,\n",
       " 735: 9,\n",
       " 736: 36,\n",
       " 733: 16,\n",
       " 732: 8,\n",
       " 730: 20,\n",
       " 737: 5,\n",
       " 740: 6,\n",
       " 738: 4,\n",
       " 739: 8,\n",
       " 741: 15,\n",
       " 743: 4,\n",
       " 742: 4,\n",
       " 745: 5,\n",
       " 744: 18,\n",
       " 746: 5,\n",
       " 747: 16,\n",
       " 748: 1,\n",
       " 749: 1,\n",
       " 750: 5,\n",
       " 753: 23,\n",
       " 752: 12,\n",
       " 751: 12,\n",
       " 754: 8,\n",
       " 755: 10,\n",
       " 759: 13,\n",
       " 761: 1,\n",
       " 758: 7,\n",
       " 764: 30,\n",
       " 757: 2,\n",
       " 763: 6,\n",
       " 760: 5,\n",
       " 765: 34,\n",
       " 756: 2,\n",
       " 762: 4,\n",
       " 769: 9,\n",
       " 770: 12,\n",
       " 771: 1,\n",
       " 768: 43,\n",
       " 767: 2,\n",
       " 766: 4,\n",
       " 773: 1,\n",
       " 777: 2,\n",
       " 775: 2,\n",
       " 774: 6,\n",
       " 772: 4,\n",
       " 776: 8,\n",
       " 778: 1,\n",
       " 783: 7,\n",
       " 781: 8,\n",
       " 780: 3,\n",
       " 779: 4,\n",
       " 785: 17,\n",
       " 784: 25,\n",
       " 782: 11,\n",
       " 786: 2,\n",
       " 787: 26,\n",
       " 791: 1,\n",
       " 788: 34,\n",
       " 798: 7,\n",
       " 790: 2,\n",
       " 793: 8,\n",
       " 792: 5,\n",
       " 789: 8,\n",
       " 797: 13,\n",
       " 796: 17,\n",
       " 794: 1,\n",
       " 795: 17,\n",
       " 799: 4,\n",
       " 801: 17,\n",
       " 804: 38,\n",
       " 802: 4,\n",
       " 800: 9,\n",
       " 803: 15,\n",
       " 805: 25,\n",
       " 810: 3,\n",
       " 807: 3,\n",
       " 809: 7,\n",
       " 808: 12,\n",
       " 806: 2,\n",
       " 812: 2,\n",
       " 811: 4,\n",
       " 813: 5,\n",
       " 815: 12,\n",
       " 814: 17,\n",
       " 817: 6,\n",
       " 816: 3,\n",
       " 820: 33,\n",
       " 821: 13,\n",
       " 822: 1,\n",
       " 818: 28,\n",
       " 824: 3,\n",
       " 825: 1,\n",
       " 819: 2,\n",
       " 823: 1,\n",
       " 826: 4,\n",
       " 827: 17,\n",
       " 834: 1,\n",
       " 833: 19,\n",
       " 830: 1,\n",
       " 829: 8,\n",
       " 832: 1,\n",
       " 835: 6,\n",
       " 831: 7,\n",
       " 828: 9,\n",
       " 836: 16,\n",
       " 839: 6,\n",
       " 840: 5,\n",
       " 838: 3,\n",
       " 842: 7,\n",
       " 841: 1,\n",
       " 837: 1,\n",
       " 845: 2,\n",
       " 844: 8,\n",
       " 843: 8,\n",
       " 854: 6,\n",
       " 847: 2,\n",
       " 851: 3,\n",
       " 853: 17,\n",
       " 852: 7,\n",
       " 848: 16,\n",
       " 846: 2,\n",
       " 850: 2,\n",
       " 849: 1,\n",
       " 855: 2,\n",
       " 856: 3,\n",
       " 858: 2,\n",
       " 857: 6,\n",
       " 860: 3,\n",
       " 861: 12,\n",
       " 859: 12,\n",
       " 862: 7,\n",
       " 864: 1,\n",
       " 869: 14,\n",
       " 863: 14,\n",
       " 866: 7,\n",
       " 867: 2,\n",
       " 865: 3,\n",
       " 870: 1,\n",
       " 868: 13,\n",
       " 871: 8,\n",
       " 878: 9,\n",
       " 875: 5,\n",
       " 877: 1,\n",
       " 874: 2,\n",
       " 873: 24,\n",
       " 879: 13,\n",
       " 876: 11,\n",
       " 872: 2,\n",
       " 882: 13,\n",
       " 880: 4,\n",
       " 881: 4,\n",
       " 883: 2,\n",
       " 884: 3,\n",
       " 885: 8,\n",
       " 886: 4,\n",
       " 887: 3,\n",
       " 889: 12,\n",
       " 888: 13,\n",
       " 891: 5,\n",
       " 890: 8,\n",
       " 892: 1,\n",
       " 893: 6,\n",
       " 894: 9,\n",
       " 898: 8,\n",
       " 896: 8,\n",
       " 895: 4,\n",
       " 899: 6,\n",
       " 897: 1,\n",
       " 903: 3,\n",
       " 902: 12,\n",
       " 901: 10,\n",
       " 900: 2,\n",
       " 907: 12,\n",
       " 904: 3,\n",
       " 906: 4,\n",
       " 905: 16,\n",
       " 908: 18,\n",
       " 910: 5,\n",
       " 909: 1,\n",
       " 911: 10,\n",
       " 913: 6,\n",
       " 912: 1,\n",
       " 915: 4,\n",
       " 916: 2,\n",
       " 914: 4,\n",
       " 917: 5,\n",
       " 918: 18,\n",
       " 919: 78,\n",
       " 921: 11,\n",
       " 920: 7,\n",
       " 922: 3,\n",
       " 924: 3,\n",
       " 923: 3,\n",
       " 925: 2,\n",
       " 926: 13,\n",
       " 929: 3,\n",
       " 930: 2,\n",
       " 928: 1,\n",
       " 927: 7,\n",
       " 932: 3,\n",
       " 933: 1,\n",
       " 931: 16,\n",
       " 934: 11,\n",
       " 937: 2,\n",
       " 936: 35,\n",
       " 935: 13,\n",
       " 939: 31,\n",
       " 938: 9,\n",
       " 942: 6,\n",
       " 940: 22,\n",
       " 943: 2,\n",
       " 941: 13,\n",
       " 944: 1,\n",
       " 946: 7,\n",
       " 945: 1,\n",
       " 950: 16,\n",
       " 954: 9,\n",
       " 951: 1,\n",
       " 947: 22,\n",
       " 948: 7,\n",
       " 952: 4,\n",
       " 955: 3,\n",
       " 953: 5,\n",
       " 949: 6,\n",
       " 960: 10,\n",
       " 959: 1,\n",
       " 958: 8,\n",
       " 956: 7,\n",
       " 957: 1,\n",
       " 964: 22,\n",
       " 962: 2,\n",
       " 965: 8,\n",
       " 961: 2,\n",
       " 963: 8,\n",
       " 966: 17,\n",
       " 968: 2,\n",
       " 967: 3,\n",
       " 973: 7,\n",
       " 970: 17,\n",
       " 969: 4,\n",
       " 972: 4,\n",
       " 971: 8,\n",
       " 975: 21,\n",
       " 978: 7,\n",
       " 979: 7,\n",
       " 980: 3,\n",
       " 974: 1,\n",
       " 976: 11,\n",
       " 977: 7,\n",
       " 981: 29,\n",
       " 983: 1,\n",
       " 985: 2,\n",
       " 982: 7,\n",
       " 984: 16,\n",
       " 986: 12,\n",
       " 987: 10,\n",
       " 989: 3,\n",
       " 988: 18,\n",
       " 991: 1,\n",
       " 990: 14,\n",
       " 992: 5,\n",
       " 995: 6,\n",
       " 997: 1,\n",
       " 994: 11,\n",
       " 996: 34,\n",
       " 998: 6,\n",
       " 993: 6,\n",
       " 999: 6,\n",
       " ...}"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dictionary.cfs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "8f_mMaGqPIo6",
    "outputId": "8dbf3054-a09d-4166-dd87-826e286f5707"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{15: 353,\n",
       " 17: 1896,\n",
       " 23: 136,\n",
       " 21: 100,\n",
       " 2: 337,\n",
       " 13: 77,\n",
       " 18: 1075,\n",
       " 19: 30,\n",
       " 1: 1303,\n",
       " 4: 69,\n",
       " 24: 23,\n",
       " 8: 359,\n",
       " 16: 48,\n",
       " 11: 99,\n",
       " 20: 9,\n",
       " 14: 183,\n",
       " 12: 829,\n",
       " 10: 1,\n",
       " 7: 131,\n",
       " 22: 91,\n",
       " 6: 7,\n",
       " 3: 71,\n",
       " 0: 314,\n",
       " 9: 10,\n",
       " 5: 11,\n",
       " 25: 26,\n",
       " 33: 551,\n",
       " 30: 7,\n",
       " 29: 21,\n",
       " 28: 32,\n",
       " 26: 180,\n",
       " 27: 10,\n",
       " 31: 60,\n",
       " 32: 109,\n",
       " 35: 110,\n",
       " 36: 79,\n",
       " 34: 19,\n",
       " 41: 156,\n",
       " 37: 139,\n",
       " 43: 8,\n",
       " 42: 603,\n",
       " 38: 195,\n",
       " 39: 61,\n",
       " 40: 5,\n",
       " 44: 301,\n",
       " 46: 4,\n",
       " 45: 28,\n",
       " 59: 507,\n",
       " 62: 545,\n",
       " 55: 4,\n",
       " 61: 236,\n",
       " 47: 118,\n",
       " 49: 331,\n",
       " 60: 24,\n",
       " 50: 366,\n",
       " 56: 1,\n",
       " 57: 44,\n",
       " 53: 6,\n",
       " 52: 12,\n",
       " 58: 4,\n",
       " 48: 21,\n",
       " 51: 2,\n",
       " 54: 73,\n",
       " 66: 311,\n",
       " 65: 11,\n",
       " 67: 376,\n",
       " 64: 22,\n",
       " 68: 10,\n",
       " 63: 114,\n",
       " 69: 215,\n",
       " 70: 91,\n",
       " 78: 73,\n",
       " 79: 30,\n",
       " 75: 98,\n",
       " 72: 115,\n",
       " 77: 36,\n",
       " 80: 317,\n",
       " 73: 240,\n",
       " 76: 577,\n",
       " 74: 2,\n",
       " 71: 136,\n",
       " 83: 179,\n",
       " 82: 85,\n",
       " 85: 5,\n",
       " 84: 136,\n",
       " 81: 8,\n",
       " 87: 99,\n",
       " 86: 67,\n",
       " 99: 64,\n",
       " 93: 3,\n",
       " 105: 10,\n",
       " 89: 3,\n",
       " 100: 79,\n",
       " 88: 1,\n",
       " 109: 213,\n",
       " 102: 3,\n",
       " 104: 2,\n",
       " 94: 26,\n",
       " 98: 31,\n",
       " 108: 350,\n",
       " 103: 58,\n",
       " 95: 10,\n",
       " 92: 9,\n",
       " 97: 2,\n",
       " 101: 3,\n",
       " 106: 510,\n",
       " 91: 1,\n",
       " 110: 144,\n",
       " 90: 2,\n",
       " 107: 113,\n",
       " 96: 7,\n",
       " 112: 37,\n",
       " 113: 21,\n",
       " 117: 171,\n",
       " 114: 6,\n",
       " 119: 120,\n",
       " 123: 28,\n",
       " 120: 10,\n",
       " 111: 78,\n",
       " 115: 14,\n",
       " 121: 19,\n",
       " 122: 185,\n",
       " 116: 79,\n",
       " 118: 60,\n",
       " 128: 121,\n",
       " 125: 289,\n",
       " 127: 101,\n",
       " 124: 81,\n",
       " 126: 104,\n",
       " 130: 20,\n",
       " 135: 7,\n",
       " 134: 27,\n",
       " 131: 85,\n",
       " 136: 43,\n",
       " 133: 88,\n",
       " 132: 20,\n",
       " 129: 1,\n",
       " 142: 17,\n",
       " 139: 12,\n",
       " 143: 23,\n",
       " 140: 9,\n",
       " 141: 4,\n",
       " 137: 151,\n",
       " 138: 7,\n",
       " 150: 173,\n",
       " 145: 9,\n",
       " 152: 78,\n",
       " 146: 23,\n",
       " 149: 4,\n",
       " 148: 2,\n",
       " 151: 3,\n",
       " 147: 206,\n",
       " 144: 44,\n",
       " 155: 13,\n",
       " 158: 603,\n",
       " 157: 109,\n",
       " 163: 27,\n",
       " 159: 9,\n",
       " 167: 23,\n",
       " 153: 6,\n",
       " 166: 21,\n",
       " 160: 30,\n",
       " 156: 18,\n",
       " 164: 51,\n",
       " 161: 14,\n",
       " 165: 9,\n",
       " 162: 36,\n",
       " 154: 20,\n",
       " 177: 46,\n",
       " 183: 5,\n",
       " 175: 14,\n",
       " 180: 26,\n",
       " 182: 99,\n",
       " 179: 1,\n",
       " 174: 52,\n",
       " 170: 2,\n",
       " 168: 73,\n",
       " 172: 2,\n",
       " 178: 260,\n",
       " 173: 36,\n",
       " 176: 16,\n",
       " 181: 177,\n",
       " 169: 2,\n",
       " 171: 2,\n",
       " 190: 19,\n",
       " 186: 21,\n",
       " 185: 9,\n",
       " 195: 102,\n",
       " 194: 1,\n",
       " 192: 10,\n",
       " 184: 91,\n",
       " 191: 38,\n",
       " 188: 143,\n",
       " 193: 62,\n",
       " 187: 157,\n",
       " 189: 51,\n",
       " 198: 163,\n",
       " 196: 33,\n",
       " 197: 84,\n",
       " 206: 19,\n",
       " 199: 2,\n",
       " 204: 1,\n",
       " 202: 35,\n",
       " 201: 13,\n",
       " 203: 3,\n",
       " 200: 2,\n",
       " 205: 22,\n",
       " 208: 50,\n",
       " 207: 10,\n",
       " 210: 17,\n",
       " 209: 35,\n",
       " 214: 39,\n",
       " 211: 39,\n",
       " 213: 23,\n",
       " 212: 122,\n",
       " 217: 351,\n",
       " 219: 169,\n",
       " 220: 13,\n",
       " 218: 52,\n",
       " 215: 31,\n",
       " 216: 2,\n",
       " 221: 4,\n",
       " 222: 68,\n",
       " 223: 1,\n",
       " 224: 1,\n",
       " 233: 40,\n",
       " 229: 8,\n",
       " 230: 20,\n",
       " 226: 9,\n",
       " 225: 26,\n",
       " 231: 271,\n",
       " 234: 104,\n",
       " 232: 21,\n",
       " 228: 21,\n",
       " 235: 10,\n",
       " 227: 2,\n",
       " 237: 3,\n",
       " 236: 51,\n",
       " 238: 15,\n",
       " 244: 166,\n",
       " 243: 12,\n",
       " 240: 33,\n",
       " 241: 2,\n",
       " 242: 3,\n",
       " 239: 24,\n",
       " 256: 48,\n",
       " 251: 189,\n",
       " 248: 80,\n",
       " 245: 39,\n",
       " 253: 8,\n",
       " 250: 53,\n",
       " 258: 157,\n",
       " 255: 8,\n",
       " 249: 66,\n",
       " 257: 24,\n",
       " 254: 7,\n",
       " 247: 14,\n",
       " 252: 70,\n",
       " 246: 32,\n",
       " 259: 4,\n",
       " 260: 47,\n",
       " 262: 18,\n",
       " 261: 30,\n",
       " 265: 59,\n",
       " 264: 2,\n",
       " 263: 55,\n",
       " 266: 1,\n",
       " 269: 7,\n",
       " 267: 8,\n",
       " 268: 41,\n",
       " 273: 261,\n",
       " 271: 115,\n",
       " 276: 1,\n",
       " 274: 6,\n",
       " 272: 92,\n",
       " 270: 19,\n",
       " 275: 4,\n",
       " 277: 20,\n",
       " 281: 17,\n",
       " 279: 17,\n",
       " 280: 35,\n",
       " 278: 1,\n",
       " 283: 15,\n",
       " 286: 3,\n",
       " 285: 8,\n",
       " 291: 95,\n",
       " 289: 3,\n",
       " 293: 17,\n",
       " 284: 17,\n",
       " 290: 14,\n",
       " 282: 3,\n",
       " 287: 18,\n",
       " 292: 33,\n",
       " 288: 18,\n",
       " 294: 28,\n",
       " 297: 174,\n",
       " 296: 142,\n",
       " 295: 13,\n",
       " 302: 91,\n",
       " 304: 68,\n",
       " 299: 2,\n",
       " 303: 6,\n",
       " 305: 45,\n",
       " 301: 2,\n",
       " 298: 11,\n",
       " 300: 43,\n",
       " 306: 63,\n",
       " 308: 3,\n",
       " 309: 34,\n",
       " 307: 8,\n",
       " 311: 11,\n",
       " 310: 12,\n",
       " 312: 1,\n",
       " 314: 7,\n",
       " 313: 50,\n",
       " 319: 49,\n",
       " 320: 81,\n",
       " 321: 48,\n",
       " 316: 4,\n",
       " 322: 21,\n",
       " 318: 43,\n",
       " 323: 89,\n",
       " 315: 24,\n",
       " 317: 13,\n",
       " 325: 111,\n",
       " 324: 14,\n",
       " 328: 69,\n",
       " 327: 36,\n",
       " 326: 2,\n",
       " 329: 3,\n",
       " 330: 2,\n",
       " 331: 26,\n",
       " 333: 110,\n",
       " 332: 5,\n",
       " 337: 3,\n",
       " 336: 70,\n",
       " 334: 11,\n",
       " 335: 10,\n",
       " 338: 9,\n",
       " 339: 6,\n",
       " 341: 52,\n",
       " 345: 6,\n",
       " 340: 2,\n",
       " 342: 20,\n",
       " 344: 62,\n",
       " 343: 9,\n",
       " 347: 1,\n",
       " 346: 1,\n",
       " 349: 44,\n",
       " 350: 4,\n",
       " 348: 5,\n",
       " 354: 1,\n",
       " 353: 159,\n",
       " 352: 3,\n",
       " 351: 17,\n",
       " 356: 62,\n",
       " 359: 5,\n",
       " 355: 24,\n",
       " 357: 5,\n",
       " 360: 2,\n",
       " 361: 16,\n",
       " 358: 24,\n",
       " 364: 37,\n",
       " 363: 110,\n",
       " 366: 24,\n",
       " 367: 2,\n",
       " 368: 80,\n",
       " 373: 4,\n",
       " 362: 4,\n",
       " 372: 3,\n",
       " 375: 42,\n",
       " 370: 1,\n",
       " 365: 6,\n",
       " 376: 17,\n",
       " 371: 11,\n",
       " 369: 6,\n",
       " 374: 10,\n",
       " 379: 20,\n",
       " 378: 7,\n",
       " 377: 12,\n",
       " 380: 3,\n",
       " 381: 58,\n",
       " 384: 4,\n",
       " 383: 7,\n",
       " 382: 7,\n",
       " 385: 10,\n",
       " 388: 20,\n",
       " 389: 23,\n",
       " 387: 6,\n",
       " 386: 7,\n",
       " 393: 66,\n",
       " 392: 24,\n",
       " 391: 4,\n",
       " 390: 107,\n",
       " 400: 28,\n",
       " 399: 14,\n",
       " 396: 9,\n",
       " 398: 123,\n",
       " 395: 7,\n",
       " 394: 7,\n",
       " 397: 12,\n",
       " 411: 13,\n",
       " 410: 57,\n",
       " 401: 2,\n",
       " 404: 3,\n",
       " 403: 10,\n",
       " 406: 55,\n",
       " 407: 6,\n",
       " 402: 11,\n",
       " 409: 2,\n",
       " 405: 38,\n",
       " 408: 5,\n",
       " 412: 40,\n",
       " 413: 4,\n",
       " 414: 5,\n",
       " 415: 19,\n",
       " 416: 3,\n",
       " 417: 13,\n",
       " 418: 1,\n",
       " 419: 13,\n",
       " 422: 26,\n",
       " 423: 9,\n",
       " 424: 26,\n",
       " 421: 42,\n",
       " 425: 4,\n",
       " 420: 9,\n",
       " 426: 38,\n",
       " 427: 81,\n",
       " 429: 26,\n",
       " 428: 8,\n",
       " 434: 21,\n",
       " 435: 2,\n",
       " 430: 5,\n",
       " 436: 5,\n",
       " 431: 6,\n",
       " 433: 2,\n",
       " 432: 6,\n",
       " 438: 56,\n",
       " 446: 9,\n",
       " 440: 17,\n",
       " 444: 18,\n",
       " 447: 24,\n",
       " 448: 1,\n",
       " 443: 59,\n",
       " 441: 1,\n",
       " 449: 2,\n",
       " 442: 9,\n",
       " 439: 10,\n",
       " 437: 11,\n",
       " 445: 13,\n",
       " 451: 12,\n",
       " 450: 15,\n",
       " 452: 5,\n",
       " 458: 3,\n",
       " 453: 16,\n",
       " 459: 66,\n",
       " 455: 3,\n",
       " 457: 28,\n",
       " 456: 19,\n",
       " 454: 43,\n",
       " 461: 5,\n",
       " 460: 73,\n",
       " 462: 15,\n",
       " 468: 1,\n",
       " 470: 7,\n",
       " 471: 4,\n",
       " 472: 3,\n",
       " 467: 6,\n",
       " 474: 4,\n",
       " 464: 21,\n",
       " 469: 47,\n",
       " 473: 49,\n",
       " 466: 7,\n",
       " 463: 5,\n",
       " 465: 19,\n",
       " 480: 30,\n",
       " 477: 4,\n",
       " 475: 3,\n",
       " 479: 3,\n",
       " 478: 15,\n",
       " 476: 15,\n",
       " 481: 139,\n",
       " 487: 29,\n",
       " 488: 1,\n",
       " 484: 3,\n",
       " 497: 17,\n",
       " 496: 17,\n",
       " 495: 4,\n",
       " 483: 10,\n",
       " 493: 11,\n",
       " 498: 27,\n",
       " 485: 2,\n",
       " 486: 31,\n",
       " 494: 3,\n",
       " 492: 2,\n",
       " 482: 78,\n",
       " 490: 4,\n",
       " 489: 50,\n",
       " 491: 4,\n",
       " 502: 34,\n",
       " 501: 28,\n",
       " 504: 4,\n",
       " 505: 1,\n",
       " 500: 3,\n",
       " 503: 134,\n",
       " 499: 10,\n",
       " 506: 10,\n",
       " 508: 3,\n",
       " 507: 72,\n",
       " 517: 5,\n",
       " 522: 67,\n",
       " 524: 20,\n",
       " 510: 1,\n",
       " 548: 2,\n",
       " 543: 10,\n",
       " 526: 33,\n",
       " 515: 9,\n",
       " 521: 3,\n",
       " 532: 2,\n",
       " 535: 1,\n",
       " 530: 10,\n",
       " 534: 2,\n",
       " 546: 3,\n",
       " 533: 1,\n",
       " 531: 3,\n",
       " 542: 2,\n",
       " 545: 2,\n",
       " 536: 3,\n",
       " 511: 2,\n",
       " 539: 6,\n",
       " 519: 1,\n",
       " 537: 1,\n",
       " 512: 4,\n",
       " 540: 2,\n",
       " 509: 15,\n",
       " 538: 5,\n",
       " 544: 1,\n",
       " 528: 2,\n",
       " 518: 4,\n",
       " 516: 1,\n",
       " 513: 4,\n",
       " 523: 1,\n",
       " 547: 50,\n",
       " 527: 1,\n",
       " 514: 1,\n",
       " 520: 12,\n",
       " 529: 21,\n",
       " 525: 9,\n",
       " 541: 3,\n",
       " 550: 19,\n",
       " 549: 2,\n",
       " 552: 16,\n",
       " 551: 29,\n",
       " 554: 3,\n",
       " 556: 51,\n",
       " 555: 1,\n",
       " 559: 5,\n",
       " 558: 53,\n",
       " 553: 10,\n",
       " 557: 3,\n",
       " 563: 4,\n",
       " 565: 82,\n",
       " 562: 26,\n",
       " 564: 5,\n",
       " 561: 22,\n",
       " 560: 20,\n",
       " 566: 10,\n",
       " 571: 8,\n",
       " 569: 3,\n",
       " 572: 30,\n",
       " 570: 4,\n",
       " 567: 4,\n",
       " 568: 4,\n",
       " 573: 120,\n",
       " 574: 21,\n",
       " 575: 13,\n",
       " 577: 26,\n",
       " 576: 29,\n",
       " 578: 19,\n",
       " 580: 2,\n",
       " 579: 3,\n",
       " 581: 7,\n",
       " 583: 1,\n",
       " 582: 16,\n",
       " 584: 33,\n",
       " 586: 12,\n",
       " 585: 16,\n",
       " 587: 16,\n",
       " 588: 5,\n",
       " 592: 53,\n",
       " 593: 8,\n",
       " 589: 4,\n",
       " 590: 17,\n",
       " 594: 3,\n",
       " 591: 8,\n",
       " 596: 12,\n",
       " 595: 13,\n",
       " 598: 3,\n",
       " 597: 6,\n",
       " 600: 7,\n",
       " 599: 4,\n",
       " 602: 6,\n",
       " 601: 7,\n",
       " 606: 6,\n",
       " 605: 51,\n",
       " 604: 67,\n",
       " 603: 11,\n",
       " 613: 6,\n",
       " 607: 14,\n",
       " 611: 21,\n",
       " 610: 3,\n",
       " 609: 34,\n",
       " 608: 14,\n",
       " 612: 29,\n",
       " 618: 20,\n",
       " 614: 40,\n",
       " 617: 2,\n",
       " 616: 8,\n",
       " 615: 34,\n",
       " 622: 10,\n",
       " 619: 19,\n",
       " 625: 1,\n",
       " 621: 4,\n",
       " 620: 10,\n",
       " 624: 1,\n",
       " 623: 18,\n",
       " 626: 4,\n",
       " 628: 4,\n",
       " 629: 6,\n",
       " 627: 6,\n",
       " 633: 4,\n",
       " 634: 1,\n",
       " 630: 4,\n",
       " 632: 1,\n",
       " 635: 60,\n",
       " 631: 23,\n",
       " 636: 25,\n",
       " 639: 26,\n",
       " 637: 12,\n",
       " 638: 3,\n",
       " 640: 26,\n",
       " 641: 19,\n",
       " 643: 1,\n",
       " 642: 9,\n",
       " 644: 3,\n",
       " 650: 12,\n",
       " 645: 25,\n",
       " 648: 4,\n",
       " 646: 7,\n",
       " 647: 6,\n",
       " 649: 36,\n",
       " 653: 28,\n",
       " 651: 29,\n",
       " 654: 12,\n",
       " 652: 25,\n",
       " 660: 5,\n",
       " 658: 37,\n",
       " 659: 7,\n",
       " 655: 2,\n",
       " 657: 1,\n",
       " 656: 1,\n",
       " 661: 27,\n",
       " 663: 40,\n",
       " 662: 19,\n",
       " 666: 72,\n",
       " 664: 3,\n",
       " 665: 6,\n",
       " 667: 18,\n",
       " 671: 9,\n",
       " 668: 22,\n",
       " 669: 21,\n",
       " 670: 1,\n",
       " 672: 7,\n",
       " 673: 16,\n",
       " 674: 1,\n",
       " 678: 3,\n",
       " 677: 2,\n",
       " 676: 19,\n",
       " 675: 35,\n",
       " 679: 4,\n",
       " 683: 9,\n",
       " 681: 8,\n",
       " 680: 2,\n",
       " 684: 4,\n",
       " 682: 7,\n",
       " 685: 24,\n",
       " 691: 23,\n",
       " 688: 2,\n",
       " 692: 5,\n",
       " 693: 7,\n",
       " 686: 2,\n",
       " 689: 3,\n",
       " 687: 5,\n",
       " 690: 1,\n",
       " 695: 26,\n",
       " 694: 8,\n",
       " 698: 23,\n",
       " 696: 5,\n",
       " 699: 19,\n",
       " 702: 3,\n",
       " 701: 5,\n",
       " 700: 15,\n",
       " 697: 11,\n",
       " 704: 7,\n",
       " 703: 2,\n",
       " 707: 8,\n",
       " 706: 38,\n",
       " 705: 34,\n",
       " 710: 7,\n",
       " 709: 4,\n",
       " 712: 18,\n",
       " 711: 67,\n",
       " 708: 11,\n",
       " 713: 11,\n",
       " 714: 6,\n",
       " 719: 14,\n",
       " 723: 2,\n",
       " 725: 1,\n",
       " 728: 4,\n",
       " 727: 8,\n",
       " 722: 1,\n",
       " 718: 4,\n",
       " 724: 33,\n",
       " 716: 1,\n",
       " 729: 6,\n",
       " 715: 3,\n",
       " 720: 6,\n",
       " 721: 1,\n",
       " 726: 11,\n",
       " 717: 2,\n",
       " 734: 4,\n",
       " 731: 17,\n",
       " 735: 9,\n",
       " 736: 36,\n",
       " 733: 16,\n",
       " 732: 8,\n",
       " 730: 19,\n",
       " 737: 5,\n",
       " 740: 6,\n",
       " 738: 4,\n",
       " 739: 8,\n",
       " 741: 15,\n",
       " 743: 4,\n",
       " 742: 4,\n",
       " 745: 5,\n",
       " 744: 18,\n",
       " 746: 5,\n",
       " 747: 16,\n",
       " 748: 1,\n",
       " 749: 1,\n",
       " 750: 5,\n",
       " 753: 23,\n",
       " 752: 12,\n",
       " 751: 12,\n",
       " 754: 8,\n",
       " 755: 10,\n",
       " 759: 13,\n",
       " 761: 1,\n",
       " 758: 7,\n",
       " 764: 28,\n",
       " 757: 2,\n",
       " 763: 6,\n",
       " 760: 5,\n",
       " 765: 33,\n",
       " 756: 2,\n",
       " 762: 4,\n",
       " 769: 9,\n",
       " 770: 12,\n",
       " 771: 1,\n",
       " 768: 41,\n",
       " 767: 2,\n",
       " 766: 4,\n",
       " 773: 1,\n",
       " 777: 2,\n",
       " 775: 2,\n",
       " 774: 6,\n",
       " 772: 4,\n",
       " 776: 8,\n",
       " 778: 1,\n",
       " 783: 7,\n",
       " 781: 8,\n",
       " 780: 3,\n",
       " 779: 4,\n",
       " 785: 17,\n",
       " 784: 25,\n",
       " 782: 11,\n",
       " 786: 2,\n",
       " 787: 26,\n",
       " 791: 1,\n",
       " 788: 33,\n",
       " 798: 7,\n",
       " 790: 2,\n",
       " 793: 8,\n",
       " 792: 5,\n",
       " 789: 8,\n",
       " 797: 13,\n",
       " 796: 17,\n",
       " 794: 1,\n",
       " 795: 17,\n",
       " 799: 4,\n",
       " 801: 17,\n",
       " 804: 36,\n",
       " 802: 4,\n",
       " 800: 8,\n",
       " 803: 15,\n",
       " 805: 25,\n",
       " 810: 3,\n",
       " 807: 3,\n",
       " 809: 7,\n",
       " 808: 12,\n",
       " 806: 2,\n",
       " 812: 2,\n",
       " 811: 4,\n",
       " 813: 5,\n",
       " 815: 12,\n",
       " 814: 16,\n",
       " 817: 6,\n",
       " 816: 3,\n",
       " 820: 33,\n",
       " 821: 13,\n",
       " 822: 1,\n",
       " 818: 28,\n",
       " 824: 3,\n",
       " 825: 1,\n",
       " 819: 1,\n",
       " 823: 1,\n",
       " 826: 4,\n",
       " 827: 17,\n",
       " 834: 1,\n",
       " 833: 18,\n",
       " 830: 1,\n",
       " 829: 7,\n",
       " 832: 1,\n",
       " 835: 6,\n",
       " 831: 7,\n",
       " 828: 8,\n",
       " 836: 16,\n",
       " 839: 6,\n",
       " 840: 5,\n",
       " 838: 3,\n",
       " 842: 7,\n",
       " 841: 1,\n",
       " 837: 1,\n",
       " 845: 2,\n",
       " 844: 8,\n",
       " 843: 8,\n",
       " 854: 6,\n",
       " 847: 2,\n",
       " 851: 2,\n",
       " 853: 17,\n",
       " 852: 7,\n",
       " 848: 16,\n",
       " 846: 2,\n",
       " 850: 2,\n",
       " 849: 1,\n",
       " 855: 2,\n",
       " 856: 3,\n",
       " 858: 2,\n",
       " 857: 5,\n",
       " 860: 3,\n",
       " 861: 12,\n",
       " 859: 12,\n",
       " 862: 7,\n",
       " 864: 1,\n",
       " 869: 14,\n",
       " 863: 14,\n",
       " 866: 7,\n",
       " 867: 2,\n",
       " 865: 3,\n",
       " 870: 1,\n",
       " 868: 13,\n",
       " 871: 8,\n",
       " 878: 9,\n",
       " 875: 5,\n",
       " 877: 1,\n",
       " 874: 2,\n",
       " 873: 24,\n",
       " 879: 13,\n",
       " 876: 11,\n",
       " 872: 2,\n",
       " 882: 13,\n",
       " 880: 4,\n",
       " 881: 4,\n",
       " 883: 2,\n",
       " 884: 3,\n",
       " 885: 8,\n",
       " 886: 4,\n",
       " 887: 3,\n",
       " 889: 10,\n",
       " 888: 13,\n",
       " 891: 4,\n",
       " 890: 8,\n",
       " 892: 1,\n",
       " 893: 6,\n",
       " 894: 9,\n",
       " 898: 8,\n",
       " 896: 8,\n",
       " 895: 4,\n",
       " 899: 6,\n",
       " 897: 1,\n",
       " 903: 3,\n",
       " 902: 12,\n",
       " 901: 10,\n",
       " 900: 2,\n",
       " 907: 12,\n",
       " 904: 3,\n",
       " 906: 4,\n",
       " 905: 16,\n",
       " 908: 16,\n",
       " 910: 5,\n",
       " 909: 1,\n",
       " 911: 8,\n",
       " 913: 5,\n",
       " 912: 1,\n",
       " 915: 4,\n",
       " 916: 2,\n",
       " 914: 4,\n",
       " 917: 5,\n",
       " 918: 18,\n",
       " 919: 74,\n",
       " 921: 11,\n",
       " 920: 7,\n",
       " 922: 3,\n",
       " 924: 3,\n",
       " 923: 3,\n",
       " 925: 2,\n",
       " 926: 13,\n",
       " 929: 3,\n",
       " 930: 2,\n",
       " 928: 1,\n",
       " 927: 6,\n",
       " 932: 3,\n",
       " 933: 1,\n",
       " 931: 16,\n",
       " 934: 11,\n",
       " 937: 2,\n",
       " 936: 34,\n",
       " 935: 13,\n",
       " 939: 31,\n",
       " 938: 9,\n",
       " 942: 6,\n",
       " 940: 22,\n",
       " 943: 2,\n",
       " 941: 13,\n",
       " 944: 1,\n",
       " 946: 7,\n",
       " 945: 1,\n",
       " 950: 15,\n",
       " 954: 9,\n",
       " 951: 1,\n",
       " 947: 22,\n",
       " 948: 7,\n",
       " 952: 4,\n",
       " 955: 3,\n",
       " 953: 5,\n",
       " 949: 6,\n",
       " 960: 10,\n",
       " 959: 1,\n",
       " 958: 8,\n",
       " 956: 7,\n",
       " 957: 1,\n",
       " 964: 22,\n",
       " 962: 2,\n",
       " 965: 8,\n",
       " 961: 2,\n",
       " 963: 8,\n",
       " 966: 17,\n",
       " 968: 2,\n",
       " 967: 3,\n",
       " 973: 7,\n",
       " 970: 17,\n",
       " 969: 4,\n",
       " 972: 4,\n",
       " 971: 8,\n",
       " 975: 20,\n",
       " 978: 7,\n",
       " 979: 7,\n",
       " 980: 3,\n",
       " 974: 1,\n",
       " 976: 11,\n",
       " 977: 7,\n",
       " 981: 29,\n",
       " 983: 1,\n",
       " 985: 2,\n",
       " 982: 7,\n",
       " 984: 15,\n",
       " 986: 12,\n",
       " 987: 10,\n",
       " 989: 3,\n",
       " 988: 18,\n",
       " 991: 1,\n",
       " 990: 13,\n",
       " 992: 5,\n",
       " 995: 5,\n",
       " 997: 1,\n",
       " 994: 11,\n",
       " 996: 32,\n",
       " 998: 6,\n",
       " 993: 6,\n",
       " 999: 6,\n",
       " ...}"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dictionary.dfs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "D8trSPV_PIo6",
    "outputId": "fe579f3a-ab24-4f32-9216-f1a8df5cf6c7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['follow', 'the', 'white', 'rabbit']\n",
      "[53, 17, 804, 327]\n"
     ]
    }
   ],
   "source": [
    "sentence = 'follow the white rabbit'\n",
    "new_tokens = simple_preprocess(sentence)\n",
    "ids = dictionary.doc2idx(new_tokens)\n",
    "print(new_tokens)\n",
    "print(ids)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "id": "jTnw8JNXPIo6"
   },
   "outputs": [],
   "source": [
    "special_tokens = {'[PAD]': 0, '[UNK]': 1}\n",
    "dictionary.patch_with_special_tokens(special_tokens)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "id": "2y0NKpAuPIo7"
   },
   "outputs": [],
   "source": [
    "def get_rare_ids(dictionary, min_freq):\n",
    "    rare_ids = [t[0] for t in dictionary.cfs.items() if t[1] < min_freq]\n",
    "    return rare_ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "id": "rO9cRL8fPIo7"
   },
   "outputs": [],
   "source": [
    "def make_vocab(sentences, folder=None, special_tokens=None, vocab_size=None, min_freq=None):\n",
    "    if folder is not None:\n",
    "        if not os.path.exists(folder):\n",
    "            os.mkdir(folder)\n",
    "\n",
    "    # tokenizes the sentences and create a Dictionary\n",
    "    tokens = [simple_preprocess(sent) for sent in sentences]\n",
    "    dictionary = corpora.Dictionary(tokens)\n",
    "    # keeps only the most frequent words (vocab size)\n",
    "    if vocab_size is not None:\n",
    "        dictionary.filter_extremes(keep_n=vocab_size)\n",
    "    # removes rare words (in case the vocab size still\n",
    "    # includes words with low frequency)\n",
    "    if min_freq is not None:\n",
    "        rare_tokens = get_rare_ids(dictionary, min_freq)\n",
    "        dictionary.filter_tokens(bad_ids=rare_tokens)\n",
    "    # gets the whole list of tokens and frequencies\n",
    "    items = dictionary.cfs.items()\n",
    "    # sorts the tokens in descending order\n",
    "    words = [dictionary[t[0]] for t in sorted(dictionary.cfs.items(), key=lambda t: -t[1])]\n",
    "    # prepends special tokens, if any\n",
    "    if special_tokens is not None:\n",
    "        to_add = []\n",
    "        for special_token in special_tokens:\n",
    "            if special_token not in words:\n",
    "                to_add.append(special_token)\n",
    "        words = to_add + words\n",
    "\n",
    "    with open(os.path.join(folder, 'vocab.txt'), 'w') as f:\n",
    "        for word in words:\n",
    "            f.write(f'{word}\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "id": "3x1exiX0PIo7"
   },
   "outputs": [],
   "source": [
    "make_vocab(train_dataset['sentence'], 'our_vocab/', special_tokens=['[PAD]', '[UNK]', '[SEP]', '[CLS]', '[MASK]'], min_freq=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7Cq5LmV6PIo7"
   },
   "source": [
    "## HugginFace's Tokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "id": "AbW7Cd8jPIo8"
   },
   "outputs": [],
   "source": [
    "from transformers import BertTokenizer\n",
    "\n",
    "tokenizer = BertTokenizer('our_vocab/vocab.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "nhxCrkJjPIo8",
    "outputId": "845b3613-0cf7-4261-ae0c-00e3ac1ca347"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['follow', 'the', 'white', 'rabbit', '[UNK]']"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_sentence = 'follow the white rabbit neo'\n",
    "new_tokens = tokenizer.tokenize(new_sentence)\n",
    "new_tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "V6DlCKGxPIo8",
    "outputId": "67da7f20-4d9a-4b5f-ad0a-8338aa43d56c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[871, 5, 206, 202, 1]"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_ids = tokenizer.convert_tokens_to_ids(new_tokens)\n",
    "new_ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "06gmmhDvPIo9",
    "outputId": "83582b16-7118-4560-b37d-27312571294e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[3, 871, 5, 206, 202, 1, 2]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_ids = tokenizer.encode(new_sentence)\n",
    "new_ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "98Ccyt_LPIo9",
    "outputId": "1f2577db-d3bc-40c3-d49a-2f69e85271ab"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['[CLS]', 'follow', 'the', 'white', 'rabbit', '[UNK]', '[SEP]']"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.convert_ids_to_tokens(new_ids)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "yVOy3y3SPIo9",
    "outputId": "7c5d5fb6-fb62-46a0-81e7-6bf3c2502f32"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[871, 5, 206, 202, 1]"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.encode(new_sentence, add_special_tokens=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "tGBDu8aoPIo9",
    "outputId": "f704d56f-62ca-4cb5-8ec2-b33269277b82"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[871,   5, 206, 202,   1]]), 'token_type_ids': tensor([[0, 0, 0, 0, 0]]), 'attention_mask': tensor([[1, 1, 1, 1, 1]])}"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer(new_sentence, add_special_tokens=False, return_tensors='pt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "VB8-t1puPIo_",
    "outputId": "3aec105b-916d-44ac-9eda-9c10a48b0d9d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': [3, 871, 5, 206, 202, 1, 2, 51, 44, 73, 34, 267, 45, 5, 1, 30, 2], 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence1 = 'follow the white rabbit neo'\n",
    "sentence2 = 'no one can be told what the matrix is'\n",
    "joined_sentences = tokenizer(sentence1, sentence2)\n",
    "joined_sentences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IjJ5EQqfPIo_",
    "outputId": "ad53d7d7-1e69-4096-9f5b-61b20579c4ca"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['[CLS]', 'follow', 'the', 'white', 'rabbit', '[UNK]', '[SEP]', 'no', 'one', 'can', 'be', 'told', 'what', 'the', '[UNK]', 'is', '[SEP]']\n"
     ]
    }
   ],
   "source": [
    "print(tokenizer.convert_ids_to_tokens(joined_sentences['input_ids']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "jNHKY9CrPIo_",
    "outputId": "023667cf-1bcf-409c-89a4-b53f236dd2fb"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': [[3, 871, 5, 206, 202, 1, 2, 0, 0, 0, 0], [3, 51, 44, 73, 34, 267, 45, 5, 1, 30, 2]], 'token_type_ids': [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]}"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "separate_sentences = tokenizer([sentence1, sentence2], padding=True)\n",
    "separate_sentences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ZOQDFqKbPIpF",
    "outputId": "61375f58-1d47-4bfd-aabe-2a0d2179a1ce"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['[CLS]', 'follow', 'the', 'white', 'rabbit', '[UNK]', '[SEP]', '[PAD]', '[PAD]', '[PAD]', '[PAD]']\n",
      "[1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0]\n"
     ]
    }
   ],
   "source": [
    "print(tokenizer.convert_ids_to_tokens(separate_sentences['input_ids'][0]))\n",
    "print(separate_sentences['attention_mask'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Uqsq7GS3PIpG",
    "outputId": "36231e21-6466-4949-c90f-5f93dc96e4f7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['[CLS]', 'follow', 'the', 'white', 'rabbit', '[UNK]', '[SEP]', 'no', 'one', 'can', 'be', 'told', 'what', 'the', '[UNK]', 'is', '[SEP]']\n",
      "['[CLS]', 'another', 'first', 'sentence', '[SEP]', '[UNK]', 'second', 'sentence', 'here', '[SEP]']\n"
     ]
    }
   ],
   "source": [
    "first_sentences = [sentence1, 'another first sentence']\n",
    "second_sentences = [sentence2, 'a second sentence here']\n",
    "batch_of_pairs = tokenizer(first_sentences, second_sentences)\n",
    "first_input = tokenizer.convert_ids_to_tokens(batch_of_pairs['input_ids'][0])\n",
    "second_input = tokenizer.convert_ids_to_tokens(batch_of_pairs['input_ids'][1])\n",
    "print(first_input)\n",
    "print(second_input)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2U40RGXSPIpG",
    "outputId": "e7ac6236-1759-4351-a560-b20d039e60fc"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[   3,   27,    1,  ...,   98, 1016,    2],\n",
       "        [   3,   23,   10,  ..., 1340,    5,    2],\n",
       "        [   3,   48,   12,  ...,    0,    0,    0],\n",
       "        ...,\n",
       "        [   3,    1,    6,  ...,    0,    0,    0],\n",
       "        [   3,    6,  127,  ...,    0,    0,    0],\n",
       "        [   3,    1,    1,  ...,    0,    0,    0]])"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenized_dataset = tokenizer(dataset['sentence'],\n",
    "                              padding=True,\n",
    "                              return_tensors='pt',\n",
    "                              max_length=50,\n",
    "                              truncation=True)\n",
    "tokenized_dataset['input_ids']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RYIpul-cPIpG"
   },
   "source": [
    "# Before Word Embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "U80QzETEPIpH"
   },
   "source": [
    "## One-Hot Encoding (OHE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NS3mlNgIPIpH"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/ohe1.png?raw=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MLGxP_hdPIpH"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/ohe2.png?raw=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-K2WB4P5PIpH"
   },
   "source": [
    "## Bag of Words (BoW)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "L6ljfjKBPIpH",
    "outputId": "85d8b612-ba91-4f7c-b15b-dc8de71c2683"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['the', 'white', 'rabbit', 'is', 'rabbit']"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence = 'the white rabbit is a rabbit'\n",
    "bow_tokens = simple_preprocess(sentence)\n",
    "bow_tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wLrldcH-PIpI",
    "outputId": "48d5b34b-4bae-407b-a3a4-d3e51871c39e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(17, 1), (273, 1), (327, 2), (804, 1)]"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bow = dictionary.doc2bow(bow_tokens)\n",
    "bow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Dfbvtiv2PIpI"
   },
   "source": [
    "## Language Models\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/blank1.png?raw=1)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/blank2.png?raw=1)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/fill1.png?raw=1)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/fill2.png?raw=1)\n",
    "\n",
    "## N-grams\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/ngrams.png?raw=1)\n",
    "\n",
    "## Continuous Bag-of-Words (CBoW)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/blank_end.png?raw=1)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/blank_center.png?raw=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UxDgKYloPIpI"
   },
   "source": [
    "# Word Embeddings\n",
    "\n",
    "## Word2Vec\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/cbow.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "id": "atT-CpiCPIpI"
   },
   "outputs": [],
   "source": [
    "class CBOW(nn.Module):\n",
    "    def __init__(self, vocab_size, embedding_size):\n",
    "        super().__init__()\n",
    "        self.embedding = nn.Embedding(vocab_size, embedding_size)\n",
    "        self.linear = nn.Linear(embedding_size, vocab_size)\n",
    "\n",
    "    def forward(self, X):\n",
    "        embeddings = self.embedding(X)\n",
    "        bow = embeddings.mean(dim=1)\n",
    "        logits = self.linear(bow)\n",
    "        return logits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xn-7a8LHPIpJ",
    "outputId": "c8ba9a57-3ee9-4c29-bbc0-c2500b180178"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "OrderedDict([('weight',\n",
       "              tensor([[ 0.3367,  0.1288,  0.2345],\n",
       "                      [ 0.2303, -1.1229, -0.1863],\n",
       "                      [ 2.2082, -0.6380,  0.4617],\n",
       "                      [ 0.2674,  0.5349,  0.8094],\n",
       "                      [ 1.1103, -1.6898, -0.9890]]))])"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.manual_seed(42)\n",
    "dummy_cbow = CBOW(vocab_size=5, embedding_size=3)\n",
    "dummy_cbow.embedding.state_dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PavMubbxPIpJ"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/w2v_embed.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "VQ4ZKqOCPIpJ",
    "outputId": "9eca9b3f-1c63-447e-97fd-6e8e83ad72b2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 2.2082, -0.6380,  0.4617],\n",
       "        [ 0.2674,  0.5349,  0.8094]], grad_fn=<EmbeddingBackward0>)"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# tokens: ['is', 'barking']\n",
    "dummy_cbow.embedding(torch.as_tensor([2, 3]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "id": "Xeo7CI0FPIpK"
   },
   "outputs": [],
   "source": [
    "tiny_vocab = ['the', 'small', 'is', 'barking', 'dog']\n",
    "context_words = ['the', 'small', 'is', 'barking']\n",
    "target_words = ['dog']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "id": "64hplr5JPIpK"
   },
   "outputs": [],
   "source": [
    "batch_context = torch.as_tensor([[0, 1, 2, 3]]).long()\n",
    "batch_target = torch.as_tensor([4]).long()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uobj-IG8PIpK"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/w2v_cbow.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hMfauJsZPIpK",
    "outputId": "a54874e2-c1fd-4b41-9daa-633590e5d308"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.7606, -0.2743,  0.3298]], grad_fn=<MeanBackward1>)"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cbow_features = dummy_cbow.embedding(batch_context).mean(dim=1)\n",
    "cbow_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "e19wHR2-PIpK"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/w2v_logits.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3e5bcIGOPIpL",
    "outputId": "772b0997-04ef-411b-c7e3-63737d03af7b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.3542,  0.6937, -0.2028, -0.5873,  0.2099]],\n",
       "       grad_fn=<AddmmBackward0>)"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logits = dummy_cbow.linear(cbow_features)\n",
    "logits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BY_RwBJSPIpL"
   },
   "source": [
    "## What is an Embeddings Anyway?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9u0vTklVPIpL"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/rest_discrete.png?raw=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QUBiuMFyPIpL"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/rest_continuous.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lPzPn8PaPIpM",
    "outputId": "cfbe0968-3a13-46b1-c5cd-b73353e09340"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 1.0000, -0.4318,  0.9976, -0.2974],\n",
       "        [-0.4318,  1.0000, -0.4270,  0.3581],\n",
       "        [ 0.9976, -0.4270,  1.0000, -0.3598],\n",
       "        [-0.2974,  0.3581, -0.3598,  1.0000]])"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings = torch.as_tensor([[.7, -.4, .7],\n",
    "                           [.3, .7, -.5],\n",
    "                           [.9, -.55, .8],\n",
    "                           [-.3, .8, .34]]).float()\n",
    "sims = torch.zeros(4, 4)\n",
    "for i in range(4):\n",
    "    for j in range(4):\n",
    "        sims[i, j] = F.cosine_similarity(ratings[i], ratings[j], dim=0)\n",
    "sims"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wkxlGEraPIpM"
   },
   "source": [
    "## Pre-trained Word2Vec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Vrvd1NarPIpM"
   },
   "source": [
    "## Global Vectors (GloVe)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wl_JFk35PIpM",
    "outputId": "9c9a32dd-2dd1-4ffc-966d-ef77a9eaf457"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[==================================================] 100.0% 66.0/66.0MB downloaded\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "400000"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from gensim import downloader\n",
    "\n",
    "glove = downloader.load('glove-wiki-gigaword-50')\n",
    "\n",
    "# UPDATED\n",
    "###########################################################\n",
    "# The vocab property isn't available anymore at the newer\n",
    "# Gensim version. It was replaced by key_to_index\n",
    "# len(glove.vocab)\n",
    "len(glove.key_to_index)\n",
    "###########################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fDzgsj0bPIpN",
    "outputId": "fd2d6db3-41d2-446e-e52b-3d277233868e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.16386 ,  0.57795 , -0.59197 , -0.32446 ,  0.29762 ,  0.85151 ,\n",
       "       -0.76695 , -0.20733 ,  0.21491 , -0.51587 , -0.17517 ,  0.94459 ,\n",
       "        0.12705 , -0.33031 ,  0.75951 ,  0.44449 ,  0.16553 , -0.19235 ,\n",
       "        0.065533, -0.12394 ,  0.61446 ,  0.89784 ,  0.17413 ,  0.41149 ,\n",
       "        1.191   , -0.39461 , -0.459   ,  0.022161, -0.50843 , -0.44464 ,\n",
       "        0.68721 , -0.7167  ,  0.20835 , -0.23437 ,  0.02604 , -0.47993 ,\n",
       "        0.31873 , -0.29135 ,  0.50273 , -0.55144 , -0.066692,  0.43873 ,\n",
       "       -0.24293 , -1.0247  ,  0.029375,  0.068499,  0.25451 , -1.9663  ,\n",
       "        0.26673 ,  0.88486 ], dtype=float32)"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "glove['alice']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "id": "46hbjBCmPIpO"
   },
   "outputs": [],
   "source": [
    "synthetic_queen = glove['king'] - glove['man'] + glove['woman']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 351
    },
    "id": "kcrptYwOPIpP",
    "outputId": "f971ef2c-58de-4c9b-a4dd-8b285d0010e4"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/content/plots/chapter11.py:23: UserWarning: FixedFormatter should only be used together with FixedLocator\n",
      "  axs[i].set_yticklabels(['', word, ''])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABvIAAAEuCAYAAABcVoUHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA0pklEQVR4nO3debRddWE2/ufk3psQIMkNkJCRQaZoExuDVBlupIigSSQqDq8EhYhMxalYhtRKfwvfBWEo8lIsk9iXCFpfELSIJSgaCcGigogUBSpEkFngJiFkuLn3/P5IuSa9sSQB/H5JPp+1WIvss+/ez9lnz88ZGp2dnc0AAAAAAAAAVelXOgAAAAAAAADQlyIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAq1lg7A+nlwSXd6mmXm/fulK8vMeA1vH/BY6QhJkqXDxpWOkF88sbR0hOw1cmDpCEmSZqP8exHue3ZF6QiZsPI3pSOke8io0hGSJF1btJeOkCO/fnfpCLn8f72xdIQkSf+fXFM6QuaPObh0hCxatqp0hLxnm87SEZIkPQPbS0dIy9Lfl46QH6wYUTpC3rDdlqUjJEm2HdAoHSGp4Hyi37LO0hGSJP+5YqvSEfK7d767dITsd8aHS0eoRutbppWOkN/2DC4dIYefd2vpCEmSm0/7y9IRMvCJ/ygdITetHFM6QkYO2qJ0hCTJ8K3K39ZburKndITs0H956QhJksb9/146Qn6/+wGlI2TR8vLrxNV3P146QpLkpNxWOkL6vXV66QhptrSVjpAbH1pSOkKS5F2LFpSOkEe/c1PpCNnx6GNLR0iSNPuXuS5uNvpl2aANu5da/ojPeulpJt2Firyu8sffNJrdpSMkSRqN8jebVhVaD9ZUw3JIklSQo1TBvqYato/yr8RqNaybS1ZU8HpUsBySpLGq/BtBVlVwDFtZwY6i0axgQSRpxLJI6lgvm5XsuavYX1WQoXyC1SpYNbPquc7SEdLoquOGcA1q2G/XsL/qfKGrdIQkdewz+1Vw7VHDPYoKTu/+S/l1ooZttHyC1RrdNewryi+NCjbRLKvhhDtJI+Xf/F1+jUgV59s1HDuSOu6TNF94vnSEKs4nkqSngvsD66v8208BAAAAAACAPhR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhTa7Im/+/Plpb2/PmWee+ZLjTpgwIRMmTPgTpAIAAAAAAIC1bXZFHgAAAAAAALwWtJYOULN//dd/LR0BAAAAAACAzZQi73+w8847l44AAAAAAADAZspXa/6XRYsWZcqUKRk6dGguueSSJOv+jbwzzzwz7e3tmT9/fq6++urst99+GTFiRPbYY4+ccsopWbZsWZ9pr1q1Kuedd14mTpyY7bffPm9605ty3nnnZeHChWlvb8/xxx//J3mOAAAAAAAAvHb4RF6SJ554IoceemgeeOCBXH755Xnf+973kn9z2WWX5eabb86UKVMyefLk3Hzzzbnkkkvy7LPP5rLLLltr3BNOOCHf+MY3stNOO+XjH/94Vq5cmX/6p3/KT37yk1frKQEAAAAAAPAat9kXef/5n/+Z973vfXnuuefy//7f/8v++++/Xn83b968zJs3L7vttluSZNmyZeno6Mg3v/nNnH766Rk5cmSS5Ec/+lG+8Y1vZMKECZk7d2623HLLJMlnP/vZTJ48+VV5TgAAAAAAALz2bdZfrXnnnXfmne98Z1544YVcf/31613iJclxxx3XW+IlycCBA3PooYemp6cnd911V+/wb3zjG0mSk08+ubfES5IRI0bkuOOOe9nPAQAAAAAAgE3TZlvk/fjHP84hhxySrbbaKnPnzs3EiRM36O/XNf7o0aOTrP69vRfdc889SZK99967z/hvectbNmieAAAAAAAAbD422yLv7rvvzvPPP583vvGN2WmnnTb47wcNGtRnWEtLS5Kku7u7d9iSJUvSr1+/bLvttn3GHz58+AbPFwAAAAAAgM3DZlvkHX300fnIRz6S66+/Ph//+MezatWqV2U+gwYNSk9PT5555pk+jz311FOvyjwBAAAAAAB47dtsi7x+/frlggsuyBFHHJHrrrsuRx999KtS5o0fPz5J8u///u99HvvJT37yis8PAAAAAACATcNmW+QlSaPRyPnnn5+ZM2fmuuuuy1FHHfWKl3kf/OAHkyRnn312li1b1jv8ySefzMUXX/yKzgsAAAAAAIBNR2vpAKU1Go2cd9556devXy6//PI0m8185StfSWvrK7No9t9//3zgAx/I1VdfnX322SdTp07NihUr8q1vfSt77rlnbrzxxvTrt1n3qQAAAAAAAKzDZl/kJavLvHPPPTf9+vXLZZddlpkzZ+YrX/nKKzb9iy66KLvvvnuuvPLKXHrppRk1alSOP/74vO1tb8uNN96YQYMGvWLzAgAAAAAAYNOw2RV5HR0d6ezs7DO80WjknHPOyTnnnNM77Je//GWf8WbNmpVZs2atc9ozZszIjBkz+gxvbW3NSSedlJNOOmmt4XPmzEmS7LbbbhvyFAAAAAAAANgM+E7HP4Enn3wyzWZzrWGPPfZYzjnnnLS0tOTggw8ulAwAAAAAAIBabXafyCvhi1/8Ym666absvffeGTZsWH73u99l7ty5WbJkSU499dSMGTOmdEQAAAAAAAAqo8j7EzjwwANz33335aabbkpnZ2e22GKL/Nmf/VmOOuqofOADHygdDwAAAAAAgAop8v4EDjzwwBx44IGlYwAAAAAAAPAa4jfyAAAAAAAAoEI+kfca0a9Rbt5tFdS9zUZL6QhJkmazWTpCWguuCy+qYTkkSTPlc5TcNl9Uw/ZR/pVYrYZ1c9CACl6PCpZDkjRb+5eOkNYKjmH9K9hRNBsVLIgkzVgWSR3rZaOSPXcdu6vyIconWK2CVTOtQ9tLR0izbYvSEapRw367hv1V+5ZtpSMkqeMcr6eCa48a7lFUcHr3X8qvEzVso+UT/JeWGvYV5ZdGBZtoBtZwwp2kmQGlI1SwRtRx/Krh2JHUcZ+kseXWpSNUcT6RlLs/sDHzbXR2dpbfkgAAAAAAAIC1VNJFAwAAAAAAAGtS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVKi1dADWT2traxqNRpF5P9TZVWS+a3rdoNIJVmtZ/GTpCHnPNU+UjpBzP/jnpSMkSXZp6SwdIY/3G1o6QrbfsvyufHFXT+kISZLBbeXfn/Lciu7SETL86f8oHSFJ8rP+u5WOkFU9zdIR8pYBz5SOkGe2Gl06QpLkP55aWjpCJnffXzpCHhte/jg64sFbSkdIknzu0R1KR8jfHrBL6Qh5bvmq0hGSJBUcwnL5hP1LR8js+ReUjpD7h7yxdIQkSaFL0LXs0r/8sWOnQ88sHSFJsvDqk0tHSM8Wg0tHyJMryq+YXRWcY9ZiUP+W0hEysKX8OpEknSvLH0hHL/lN6QhZOXyP0hFy+6PPl46QJNlr1NalI+TJpeXv6y7sXF46QiZv8VTpCKv1K38Pb9X9Py0dIffsMq10hCTJ+GFbFJlvs9lM16oNuwYsv+awXhqNRrEir4bb8zVcQCZJo4KlsbiCGz3ll8JqjZS/eGqm/MpZat+wthoyWBa9CZrlLyCTpLv8JppVFWRoNGvYa5ZfL5NKXo+e8sfRKo4d3eUv6JNkaQ1vBKng2FHDOpEkPRWcWy195rnSEarYT1SwZSSp4+hRQ4ZnFpUvE5M6rn8q2GVWsc8s/0rUpPzrUcWKmTrWzRquPWq4Lq/huiOpY1nUsF7W8HrUsG0kSSrI0Vi1snSEKu4XJXVso+ur/EcXAAAAAAAAgD4UeQAAAAAAAFAhRR4AAAAAAABUSJEHAAAAAAAAFVLkAQAAAAAAQIUUeQAAAAAAAFAhRR4AAAAAAABUSJEHAAAAAAAAFVLkAQAAAAAAQIU2myJv/vz5aW9vz5lnnpnbb78906ZNy5gxY7LLLrvks5/9bJYtW5YkmTt3bt7xjndk1KhR2W233XLaaadl1apVvdNZtGhRzj///EyZMiXjxo3LsGHDMm7cuBx77LF56KGH+sz3zDPPTHt7e+bPn5+rr746++23X0aMGJE99tgjp5xySu98AQAAAAAAYE2bTZH3ojvuuCPTp0/P4MGDc+SRR2bMmDG5/PLL8+lPfzrXXnttjjjiiIwdOzZHHnlkhgwZkgsuuCD/8A//0Pv3999/f84444wMHDgw06ZNy/HHH5+JEyfmmmuuyQEHHJCHH354nfO97LLL8pnPfCavf/3r87GPfSzt7e255JJL8qlPfepP9dQBAAAAAAB4DWktHeBP7fvf/36uuuqqTJ06NUnS1dWV/fffP1dffXVuvvnmfPe7382kSZOSJLNmzcqkSZNy8cUX58QTT0xbW1t233333HfffRk6dOha073lllvynve8J+eee24uuOCCPvOdN29e5s2bl9122y1JsmzZsnR0dOSb3/xmTj/99IwcOfJVfuYAAAAAAAC8lmx2n8jr6OjoLfGSpK2tLdOnT0+z2cw73/nO3hIvSQYNGpSDDz44zz33XB599NEkyZAhQ/qUeEkyefLkjBs3LvPmzVvnfI877rjeEi9JBg4cmEMPPTQ9PT256667XpknBwAAAAAAwCZjs/tE3oQJE/oMGzFixEs+9sQTT2SnnXZKsvr39i666KLccccdeeaZZ9b6Db3+/fuvc74TJ07sM2z06NFJVv/uHgAAAAAAAKxpsyvyBg0a1GdYS0vLSz7W1dWVJPnWt76VmTNnZuutt84BBxyQHXbYIQMHDkyj0cjXvva1PPLIIxs83+7u7o17MgAAAAAAAGyyNrsi7+WaPXt2tthii8ybNy+77LLLWo9de+21hVIBAAAAAACwqdnsfiPv5XrooYey++679ynxnnjiiSxcuLBMKAAAAAAAADY5irwNNHbs2Dz00EN56qmneoctX748J554Yu/XbwIAAAAAAMDL5as1N9AxxxyTk08+OZMnT84hhxyS7u7u/PCHP0yz2cz48eNzzz33lI4IAAAAAADAJsAn8jbQ0UcfnS9+8YsZOnRo5syZk+985zvZd999873vfS9DhgwpHQ8AAAAAAIBNxGbzibyOjo50dnau87EZM2ZkxowZ63xs1qxZmTVrVu+/G41GZs6cmZkzZ/YZ94YbbnjJv1/f+QIAAAAAALB584k8AAAAAAAAqJAiDwAAAAAAACqkyAMAAAAAAIAKKfIAAAAAAACgQoo8AAAAAAAAqFBr6QCsn2azWWzeNbS9BZ/+WpoVLI3BW5TfbMsvhdWaaZSOkEbKr5wl9w9/UEMGy6I3QaOldIQkSUv5TTTNKjLUsNcsv14mSWsNr0e/8sfRKo4dLW2lIyRJtmqrYPuo4NhRwzqR1HGOt9W2Q0tHqGI/UcNrkaSCs+06jmDbDtmqdIQkdVz/VLDLrGKfWUOGelSwLCqIkNSxXtRw7VHDdXkN1x1JHcuihvWyhtejhm0jSVJBjmZr/9IRqrhflJTbRjdmvo3Ozs7yWzMAAAAAAACwlvIVMAAAAAAAANCHIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKKPAAAAAAAAKiQIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKKPAAAAAAAAKiQIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKKPAAAAAAAAKiQIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKKPAAAAAAAAKiQIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKKPAAAAAAAAKiQIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKKPAAAAAAAAKiQIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKKPAAAAAAAAKiQIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKtpQOwftoaSaNRZt63Pb68zIzX8Lb+j5WOkCRptm5ROkI+cMNzpSNkRPvA0hGSJBd0bF06Qn6yfGjpCBm2ZVvpCNlxQPn9RJI83bNl6QgZ8cSdpSNk4bCJpSMkSZ5e2lU6Qh5ZVH7dfO/I8suh54Gflo6QJPnP1x1cOkLGDi6/z+xcsap0hIxa+tvSEZIk77zqqdIR8u1P7V06QhY8srh0hGrc/pdTS0fIqTfOLh0hC0f8RekISZIt28q/93dYd2fpCPnzE68vHSFJ8ov/877SEZJmT+kEWd5/SOkIeX5V+eWQJP37FbpRtGaGlvL7ie6eZukISZLfLyt/jrfjw/NLR0hz171KR8hvlpW/f5ckK3vK7yt2GtK/dITc90z56/I3PXlr6QhJkuYe+5SOkKVXX1g6Qh5+12dKR0iSjNtmQJH5NpvNdK3asGOGIu81olGwyFtVwflQo4KLhSRVXLQsXtFdOkK2Xlk+Q5I0Un7lrGH76En5i7fyCVZrVpCk0VP+4q2G5ZAk3RVsHysruKivYV/VWLWydIQkSfmjaNIodUK1hhq20VrOrRZVcMOrhnWiq46XowrLnin/pjXH8j+oIUcNx9Fnl5S/AZnUc85dXAX77WpejQqWRQ3H0Vpejir2md3l30RYg1pOrSq4HK1iP1HDvQHbxhpWvFA6QR3bRio5hq2n8m+bAQAAAAAAAPpQ5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFihR5nZ2d2WabbfKhD31oreF333132tvb097engcffHCtx6ZOnZoRI0ZkxYoVvcOuvPLKvP3tb8/o0aMzevTovP3tb89VV13VZ37z589Pe3t7zjzzzNx+++2ZNm1axowZk1122SWf/exns2zZsiTJ3Llz8453vCOjRo3KbrvtltNOOy2rVq1aa1qLFi3K+eefnylTpmTcuHEZNmxYxo0bl2OPPTYPPfRQn3mfeeaZaW9vz/z583P11Vdnv/32y4gRI7LHHnvklFNO6Z03AAAAAAAArKlIkdfe3p7x48fnxz/+cbq7u3uHz58/f53/v3z58vzsZz/LXnvtlQEDBiRJTj755HziE5/I448/nsMPPzyHH354Hn/88Zxwwgk55ZRT1jnfO+64I9OnT8/gwYNz5JFHZsyYMbn88svz6U9/Otdee22OOOKIjB07NkceeWSGDBmSCy64IP/wD/+w1jTuv//+nHHGGRk4cGCmTZuW448/PhMnTsw111yTAw44IA8//PA6533ZZZflM5/5TF7/+tfnYx/7WNrb23PJJZfkU5/61EYvRwAAAAAAADZdraVm3NHRkbvvvjt33XVX9txzzySry7tdd901y5cvz/z583PEEUckSW6//fasWLEiHR0dSZIFCxbk0ksvzR577JGbbropQ4YMSZLMmjUrBx54YC655JJMnz49++yzz1rz/P73v5+rrroqU6dOTZJ0dXVl//33z9VXX52bb7453/3udzNp0qTeaU2aNCkXX3xxTjzxxLS1tSVJdt9999x3330ZOnToWtO+5ZZb8p73vCfnnntuLrjggj7Pd968eZk3b1522223JMmyZcvS0dGRb37zmzn99NMzcuTIV2S5AgAAAAAAsGko9ht5L5Zyt9xyS5Kku7s7t912Wzo6OrLffvut89N5++23X5Lk61//epLk1FNP7S3xktWf9Hvx03hf+9rX1jnPF0u8JGlra8v06dPTbDbzzne+s7fES5JBgwbl4IMPznPPPZdHH320d/iQIUP6lHhJMnny5IwbNy7z5s1b5/M97rjjeku8JBk4cGAOPfTQ9PT05K677lrn3wAAAAAAALD5Klbk7bPPPmlpaekt6e6+++4sXrw4kydPTkdHR5588sncd999SZJbb701AwcOzJvf/ObecZM/FHtrerEg/OUvf9nnsQkTJvQZNmLEiJd87Iknnlhr+Pz583PYYYdljz32yHbbbdf7u3733ntvn3FfNHHixD7DRo8enWT17+4BAAAAAADAmop9tebgwYPz53/+57n99tvT1dWV+fPnp9FopKOjIy+88EKS1YXZ2LFjc8cdd2TfffdN//79kyRLlixJv379st122/WZ7vDhw9NoNLJkyZI+jw0aNKjPsJaWlpd8rKurq3fYt771rcycOTNbb711DjjggOywww4ZOHBgGo1Gvva1r+WRRx5Z5/P9n6a/5u8EAgAAAAAAQFKwyEtWf3ruzjvvzB133JFbb701r3/963vLuR133DHz58/PLrvskq6urt5P2iWrS7Genp78/ve/z7Bhw9aa5tNPP51ms7nO4uyVMHv27GyxxRaZN29edtlll7Ueu/baa1+VeQIAAAAAALD5KfbVmskfvgbzBz/4QX784x+vVdZNnjw5t956a+9v6K35NZpvfOMbk6z+ys3/7sVh6/qqzFfCQw89lN13371PiffEE09k4cKFr8o8AQAAAAAA2PwULfLe+ta3prW1NV/5yleyZMmSTJ48ufexjo6OPPPMM/nqV7+arbbaKpMmTep97MMf/nCS5KyzzsrixYt7hy9atChnnXXWWuO80saOHZuHHnooTz31VO+w5cuX58QTT1zrKzgBAAAAAADg5Sj61Zpbb711Jk2alJ/85Cfp169f9t13397HXvx03u9///u8/e1vT1tbW+9j++67b4455phceuml2WefffLud787zWYz119/fR599NEce+yxa03rlXTMMcfk5JNPzuTJk3PIIYeku7s7P/zhD9NsNjN+/Pjcc889r8p8AQAAAAAA2LwU/URe8ofC7o1vfGPa29t7h48cOTK77rprkrW/VvNFZ599di688MIMHz48V1xxRebMmZPtt98+F154Ye+n8l4NRx99dL74xS9m6NChmTNnTr7zne9k3333zfe+970MGTLkVZsvAAAAAAAAm5dGZ2dns3QIXlr/fkmjUWbeP3p0eZkZr+HA/r8rHSFJ0mzdonSEHHTds6UjZNTQgaUjJEn++e2DS0fIgmXblI6Q7bfqXzpCXjdgWekISZIne7YqHSGjHru9dIQ8uP1epSMkSZ58fmXpCFnYWX7d/F+jy3/1ds+vbisdIUly3+7vLh0hOw0pv898dvmq0hEy5vkHS0dIkrz18idKR8iPTp380iO9yn64cFHpCNVYMHGf0hFy+u1fKh0hD456db5hZkNt2Vb8vb/Zvrv89c+Yo68pHSFJ8ruvHFY6QtLsKZ0gywa0l46Q57vKL4ck6d9S6EbRGga0lN9PrOqp49bm0y+UP8fb+aGbS0dIc4/yx/L7l5W/f5ckK7vL7yteN3RA6Qi59+ny1+V7Pf7D0hGSJM03vK10hDx/5dmlI2Th9L8tHSFJ8mfbldlXNJvNrNzAn2krf7QFAAAAAAAA+lDkAQAAAAAAQIUUeQAAAAAAAFAhRR4AAAAAAABUqLV0ANZPs+Dv9raW/+3kNBt1dM415Bg8oKV0hGzdv3yGJGmm/MpZw/bRL+V/2Lt8gtUaFSRp9it/aK1hOSRJSwXbR/9+5UPUsK9qtvYvHSFJHe8ga5Y8qfovNWyjNZzTJMmQgeX3mTWsE211vBxVGLjt0NIRHMvXUEOOGo6j2wzaonSEJPWccxdXwX67mlejghg1HEfrWCcq2We2tJWOUMFSqOO6I0kquBytYvuo4d5ADdtGUsf2kQFblk5Qx7aRcsewjZlvo7Ozs4r1BwAAAAAAAPiDWt6gAAAAAAAAAKxBkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUKHW0gFYP8/3NJI0isx7u+7FRea7lod+XjpBkuTmrfcsHSEH9XuwdITc2P260hGSJNsMbCsdIVv9f8eVjpCc8eXSCfLRM24sHSFJstP4saUj5P98YELpCBnRv7t0hCTJqpYBpSOkJT2lI+Se368sHSGtLWXOIf67nz26qHSEHD5heOkI6ffL75WOkOw8sXSCJEljVfntY/7zg0tHyOTu+0tHSJJ0P/1o6QhZtechpSNk8cryx9Er73q8dIQkyZ6jh5SOkI5ld5eOkKk3tZSOkCS5fp/nSkdI18R3lY6Q1p6u0hFyx9PlMyRJ+xblr4lHbV3+1uLn59ZxHD3jXXuUjpBHFpdfN7dsK3/tMXb5I6UjJElu79q+dIR0Li+/Tjy7rHyG/zVieekISZKbFg0qHSFvGrl16QgZ9kId57r9lj5bZL7NRkteGL77Bv1N+aMt66lckddIs8h819JdfoefJF3l7wenkfI3FmpYDknSXcGq2bNkSekIqWETfe758jdik2SbFeW3j2ahffWaGuUjJEkaFQRpVPB61LCv6ldBhiRZUcHCqGK9rOK8pvxrkdRxnrmqfIQ0elaVjpCkjmK1hm201HXXmpbVsGLG9vGiRcsqWBBJGt220dUZSieoY9tIUsFb1lLFC/L8yiqWRBXbRw1Loopr4mYNS6KOfUUN9/CquA6sYuuo4/Wo4Vy3ltej0Sx/H3F9+WpNAAAAAAAAqJAiDwAAAAAAACqkyAMAAAAAAIAKKfIAAAAAAACgQoo8AAAAAAAAqJAiDwAAAAAAACq0yRV5U6dOTXt7+2Y7fwAAAAAAADYNr7ki76qrrkp7e3uuuuqqIvM/88wz097envnz5xeZPwAAAAAAAJuH1tIBNjUXX3xxli1bVjoGAAAAAAAAr3GKvFfY2LFjS0cAAAAAAABgE7DBX6357W9/O1OmTMmuu+6a7bffPuPGjcv06dPz7W9/O7/5zW8ydOjQfOADH1jn3y5ZsiSjR4/OXnvt1Tvs+OOPT3t7exYuXJiLL744e+21V4YPH57x48dn9uzZ6enpWWvcE044IUlywgknpL29vfe//66rqytnnnlmJkyYkOHDh2fPPffMl7/85XXmajab+epXv5qDDz44Y8eOzciRI7P//vvnq1/96lrjTZ06NWeddVaS5N3vfnfvvCdMmLDWOH/sN/JuuOGGvPe9783OO++c7bffPhMmTMgxxxyTe++9d53jAwAAAAAAsPnaoE/kXX755fnsZz+bESNGZNq0adlmm23y5JNP5s4778wNN9yQ6dOnp6OjIzfffHN+97vfZcyYMWv9/TXXXJOlS5fmox/9aJ9pn3baaVmwYEEOPvjgHHDAAbnhhhsye/bsdHV15fOf/3yS1SXZokWL8t3vfjdTpkxZq0D774466qjceeedOfDAA9PS0pLrrrsuf/M3f5O2trYcccQRveM1m80cffTRueaaa7LLLrvk/e9/f9ra2jJv3rx88pOfzH333Zf//b//d5LksMMOS5IsWLAgH/7wh7PDDjskSYYMGfKSy+5zn/tcvvSlL2Xo0KGZOnVqhg0blkcffTQ/+tGPMnHixLzhDW94yWkAAAAAAACw+digIm/OnDnp379/5s+fn2HDhq312LPPPpskmTlzZm655ZZceeWVOfXUU9f59x/+8If7TPsXv/hFFixYkBEjRiRJTj755EyaNCmXXnppTjnllPTv3z/Tpk3rLfKmTp2aGTNm/NGsjz32WG677bYMHjw4SXLcccdl7733zoUXXrhWkTdnzpxcc801mTFjRs4///y0tbUlSVauXJmPfvSjufDCC/P+978/EydOzIwZM/Lwww9nwYIFOeyww9LR0bFey+3GG2/Ml770pbzhDW/Id77znWyzzTa9j61atap32QEAAAAAAMCLNvirNdva2nrLrjW9WE5NmzYtw4cPz1VXXbXW12Lec889+fnPf54pU6Zku+226/P3J510Um+JlyTbbrttpkyZkiVLluSBBx7Y0Jg57bTTeku8JNltt93ylre8JQ888ECWLFnSO/zSSy/NVlttlXPPPXet59W/f//eTwJec801Gzz/NV1++eVJktmzZ69V4iVJa2trhg8f/rKmDwAAAAAAwKZngz6Rd+ihh+a0007L3nvvnfe///3p6OjIW9/61rUKs7a2tsyYMSNf/OIX84Mf/CAHHnhgkuSKK65IkrU+DbemiRMn9hk2evToJMmiRYs2JOZ6TW/QoEF54YUXcu+992bkyJE5//zz+4y/atWqJNmoInFNd9xxRwYMGJD99tvvZU0HAAAAAACAzccGFXmf/OQnM3To0HzlK1/JhRdemH/8x39Ma2trDjrooJxxxhnZaaedkqwu684///zMmTMnBx54YJYvX56rr746O+64Y/bff/91TnvQoEF9hrW0tCRJuru7N+xZJWuVi39sep2dnWk2m3nsscdy1lln/dFpLV26dIPnv6bFixdn5MiR6ddvgz8ACQAAAAAAwGZqg4q8RqORj3zkI/nIRz6SZ599Nrfddlu++c1v5rrrrsuDDz6YBQsWpKWlJTvttFMOOOCA/Nu//Vuefvrp/PCHP0xnZ2c+8YlPpNFovFrPZYO9WB5OnDgx8+bNe9XmM2TIkDz11FPp6elR5gEAAAAAALBeNrpV2mabbTJt2rT88z//cyZPnpxf//rXefDBB3sfP/LII9PV1ZWvf/3rmTNnTlpaWjJjxoyXHfjlfErvvxs0aFD22GOP3H///ens7Nyg+a/5+38vZc8998yKFSty6623bkxMAAAAAAAANkMbVOTNnz8/zWZzrWFdXV157rnnkiQDBgzoHf6ud70rI0eOzD/90z9lwYIFOeiggzJy5MiXHXjo0KFJkkcfffRlTytJjj322Lzwwgv59Kc/vc6v0Fy4cGF++9vf9pn/7373u/Wex8c//vEkyamnntq7rF60atWqPPXUUxsTHQAAAAAAgE3YBn215owZMzJ48OC8+c1vztixY9PV1ZV58+bl17/+daZPn54ddtjhDxNubc3hhx+ec845J8nq3817JfzFX/xFBg4cmIsuuiidnZ3ZbrvtkiQnnXTSRk1v5syZ+elPf5qvf/3ruf322/O2t70tI0eOzFNPPZUHHnggP/vZz/LlL385O+64Y5Kko6MjjUYjX/jCF/LrX/86gwcPzpAhQ3LMMcf80XkcdNBB+eQnP5l//Md/zKRJkzJt2rQMGzYsjz32WG655ZZ84hOfyF/91V9tVH4AAAAAAAA2TRtU5P393/99vv/97+eOO+7IjTfemC233DI777xzzjvvvHzkIx/pM/5hhx2Wc845J6NGjco73vGOVyTw0KFDc8UVV2T27NmZM2dOli1blmTji7xGo5GLLrooBx10UK644orMnTs3S5cuzbBhw/K6170uX/jCF7L//vv3jj9u3Lh86UtfyoUXXphLL700K1asyNixY//HIi9JvvCFL2SvvfbKZZddlm9/+9tZsWJFtt9++3R0dOQv//IvNyo7AAAAAAAAm64NKvKOOuqoHHXUUes9/t13351k9Sf5Xvxtuf/uoosuykUXXbTOx2bNmpVZs2b1GX7QQQfloIMOWuff3HDDDX80z/80r/e+971573vf+0f/dk2HHXZYDjvssA2e/yGHHJJDDjlkveYBAAAAAADA5m2DfiNvQzSbzVx44YVpbW19xb5WEwAAAAAAADYXG/SJvPXxH//xH5k7d25uv/32/PSnP83MmTMzZsyYV3o2AAAAAAAAsEl7xYu8u+66K6effnoGDx6cD33oQ/nCF77wSs8CAAAAAAAANnmveJE3Y8aMzJgx45WeLAAAAAAAAGxWXrXfyAMAAAAAAAA23iv+iTxeLc2Cc24Um3evlrbSCZIkbRVU381+LaUjpK3c6riWlgpWzX6DBpWOkBo20aFb9y8dIUkyeED57aNRcH/9omb5CEmSZgVBmhW8HlXsqyrIkCQDKlgYVayXVZzXlH8tViufo7V8hDT71XFZ1mwtfzyvYRstee31ooE1rJixfbxoyMDy55hJ0myxja7OUDpBHdtGUsm78yt4QbbuX8WSqGL7qGFJVHFN3KhhSdSxr6jhXmYV14FVbB11vB41nOvW8no0G2XO8TZmvo3Ozs7yrxwAAAAAAACwljqqTwAAAAAAAGAtijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAq1lg7A+nlyWdJTaN7Hffknheb8B9eN+XHpCEmSLT74N6Uj5OHFXaUj5OkXymdIkj2HtZWOkH/9zZLSETJ1t21KR0j//1xQOkKS5PExbykdIf/2wDOlI+QN221dOkKS5C3tK0tHyO96yi+LJ5eW32e+efHPS0dIkizaae/SETL0uQdKR8jjg3YtHSGPLF5ROkKSZJuB5Y/lOw5qKR0hj79Q6ky/PktPPqp0hOzyoYNKR0i/fd5fOkKSpPW5h0tHyLLtdisdIYfPqeM4euZ7xpeOkNdtsbx0hDy8covSEbJsVbN0hCTJdluWP4Y9VsE9itdvV36dSJK2e39QOkK6n3m8dIQ8MOEDpSNkyIA6Pruy/YDy+4rWhXeUjpDuHd9UOkKe7Cq/v0yS0c8/VDpC/u3gvyodIfv99KbSEZIkg35/X5H5Nhv98sLQnTfobxR5rxE9KVfkLVq2qtCc17BiWekESZJGo1E6QnpSPkMl1yyp4OXIyp7yC6OG9bLRU8F+Ikmzgu1jRXf5daKabTTlg9SwTlSwSlSzjdaw4240yxcm1ss/KP9q1HEcrWGdqEXP8+XfJNXoKl90V7BaJqljn1nDNrp4RXfpCEkq2WeWDpA69pk9zUoOpBUsixrOKWrYTyRJo7t8qVnDMayC2yRV7CeSpNEovzBquBasYROtZp2o4Nyq67nO0hHqWCmSNJp1nOOtjzrengAAAAAAAACsRZEHAAAAAAAAFVLkAQAAAAAAQIUUeQAAAAAAAFAhRR4AAAAAAABUSJEHAAAAAAAAFVLkAQAAAAAAQIUUeQAAAAAAAFAhRR4AAAAAAABU6DVR5HV3d+f888/Pm970pmy//fZ505velPPOOy8LFy5Me3t7jj/++N5x29vbM3Xq1HVOZ8KECZkwYUKf4StXrsyFF16YyZMnZ9SoURkzZkze9a535bvf/e46p7Mh4x9//PFpb2/PwoULc/HFF2evvfbK8OHDM378+MyePTs9PT0buVQAAAAAAADYlLWWDrA+Pv3pT+fKK6/MjjvumI9//ONZsWJFvvSlL+X2229/2dNesWJFDj300Nx6662ZMGFCDj/88KxatSo33XRTDjvssJx99tk55phjNnr8F5122mlZsGBBDj744BxwwAG54YYbMnv27HR1deXzn//8y34eAAAAAAAAbFqqL/Lmz5+fK6+8MuPHj8/cuXOz1VZbJUlOPPHEdHR0vOzpn3322bn11ltz0kkn5W//9m/TaDSSJEuWLMkhhxySv/u7v8u73/3ujBw5cqPGf9EvfvGLLFiwICNGjEiSnHzyyZk0aVIuvfTSnHLKKenfv//Lfi4AAAAAAABsOqr/as1/+Zd/SbK6+HqxxEuSUaNG5bjjjntZ0+7p6cnll1+enXfeea1SLkkGDRqUk08+OStXrsz111+/UeOv6aSTTuot8ZJk2223zZQpU7JkyZI88MADL+t5AAAAAAAAsOmp/hN599xzT5Jkn3326fPY3nvv/bKm/cADD6SzszMjR47M7Nmz+zz+zDPP9I63MeOvaeLEiX2GjR49OkmyaNGijX4OAAAAAAAAbJqqL/IWL16cfv36Zdttt+3z2PDhw1/WtJ977rkkya9+9av86le/+qPjLV26dKPGX9OgQYP6DGtpaUmSdHd3r39oAAAAAAAANgvVF3mDBw9OT09PnnnmmWy33XZrPfbUU0/1Gb/RaPzRYmzx4sUZPHhw779fLNcOOeSQzJkz5yWzbOj4AAAAAAAAsLGq/4288ePHJ0luu+22Po/9+Mc/7jOsvb09jz32WJ/hv/3tb/t8heUee+yRwYMH5+c//3m6urpeMsuGjg8AAAAAAAAbq/oi70Mf+lCS5Oyzz17rKysfe+yxXHzxxX3GnzRpUh5++OHceuutvcNWrlyZz33uc33GbW1tzcc+9rE88sgj+bu/+7t1lnP33ntvnn766Y0aHwAAAAAAADZW9V+tOXny5MyYMSNXXXVV9tlnn0ybNi0rV67Mtddemze/+c2ZO3fuWuOfcMIJ+cEPfpAPfvCDOfTQQzNw4MDMmzcvQ4YMyYgRI/pMf9asWfnFL36RSy65JDfddFP22WefDBs2LI899ljuvffe3HPPPfne976XYcOGbdT4AAAAAAAAsDGqL/KS5IILLsiuu+6aK664IpdddllGjRqVE044Ie9973v7FHkHHHBA/u///b8566yz8o1vfCNDhw7N9OnTc9ppp2XvvffuM+0BAwbkmmuuyVe/+tX8y7/8S66//vqsWLEiw4YNy7hx4/Kxj30sb3jDGzZ6fAAAAAAAANgYr4kir6WlJX/913+dv/7rv15r+G9/+9t1jj99+vRMnz69z/Bf/vKXf3T6Rx55ZI488sj1zrO+41900UW56KKL1vnYrFmzMmvWrPWaJwAAAAAAAJuX6n8jDwAAAAAAADZHijwAAAAAAACokCIPAAAAAAAAKvSa+I28P2bHHXdMZ2dn6RgAAAAAAADwivOJPAAAAAAAAKjQa/oTeZuTko3rkIEVrCYDBpZOkCRpNpulI6RfymdobZROsFoFL0f69yu/MGpYL5v9KthPJGlUsH0MaCm/TlSzjaZ8kBrWiQpWiWq20Rp23M1G+fexWS//oPyrUcdxtIZ1ohb9th5UOkKabQNKR6hhd5mkjn1mDdvo4AEtpSMkqWSfWTpA6thnVnAZ+F/KL4sazilq2E8kSbOlrXSEKo5hNWwfNewnkjqO5zVcC9awHKpZJyo4t2ob2l46Qh0rRZJmo8w53sasB43Ozs46lhoAAAAAAADQq3wFDAAAAAAAAPShyAMAAAAAAIAKKfIAAAAAAACgQoo8AAAAAAAAqJAiDwAAAAAAACqkyAMAAAAAAIAKKfIAAAAAAACgQoo8AAAAAAAAqJAiDwAAAAAAACqkyAMAAAAAAIAKKfIAAAAAAACgQv8/8Zish2uD/N0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1800x350 with 5 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plot_word_vectors(glove,\n",
    "                        ['king', 'man', 'woman', 'synthetic', 'queen'],\n",
    "                        other={'synthetic': synthetic_queen})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "gtLfGWX7PIpP",
    "outputId": "f0baf3be-00ce-4658-d315-1b39c7783b1b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('king', 0.8859834671020508),\n",
       " ('queen', 0.8609582185745239),\n",
       " ('daughter', 0.7684512734413147),\n",
       " ('prince', 0.7640699744224548),\n",
       " ('throne', 0.7634970545768738)]"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "glove.similar_by_vector(synthetic_queen, topn=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fEk297ScPIpQ"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/embed_arithmetic.png?raw=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6uv9IVvGPIpQ"
   },
   "source": [
    "$$\n",
    "\\Large\n",
    "w_{\\text{king}} - w_{\\text{man}}\\approx w_{\\text{queen}}-w_{\\text{woman}} \\implies w_{\\text{king}} - w_{\\text{man}} + w_{\\text{woman}} \\approx w_{\\text{queen}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "O7M0z1YMPIpQ"
   },
   "source": [
    "## Using Word Embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MuAY6Gk6PIpQ"
   },
   "source": [
    "### Vocabulary Coverage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bpLS4nhMPIpQ",
    "outputId": "3c5e86c5-f7c1-45bc-9b7d-87fd759997dd"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3728"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocab = list(dictionary.token2id.keys())\n",
    "len(vocab)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "AWvfyewTPIpQ",
    "outputId": "f87df5da-f533-4199-a7d9-adafc20db2cd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "44\n",
      "['[PAD]', '[UNK]', 'arrum', 'barrowful', 'beauti']\n"
     ]
    }
   ],
   "source": [
    "# UPDATED\n",
    "###########################################################\n",
    "# The vocab property isn't available anymore at the newer\n",
    "# Gensim version. It was replaced by key_to_index\n",
    "# unknown_words = sorted(list(set(vocab).difference(set(glove.vocab))))\n",
    "\n",
    "unknown_words = sorted(list(set(vocab).difference(set(glove.key_to_index))))\n",
    "###########################################################\n",
    "print(len(unknown_words))\n",
    "print(unknown_words[:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "EddC0V1zPIpR",
    "outputId": "6707d919-7386-4291-dc7e-c54facb06f0b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(87, 50926)"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unknown_ids = [dictionary.token2id[w] for w in unknown_words if w not in ['[PAD]', '[UNK]']]\n",
    "unknown_count = np.sum([dictionary.cfs[idx] for idx in unknown_ids])\n",
    "unknown_count, dictionary.num_pos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "id": "IWUf3u6CPIpR"
   },
   "outputs": [],
   "source": [
    "def vocab_coverage(gensim_dict, pretrained_wv, special_tokens=('[PAD]', '[UNK]')):\n",
    "    vocab = list(gensim_dict.token2id.keys())\n",
    "    # UPDATED\n",
    "    ###########################################################\n",
    "    # The vocab property isn't available anymore at the newer\n",
    "    # Gensim version. It was replaced by key_to_index\n",
    "    # unknown_words = sorted(list(set(vocab).difference(set(pretrained_wv.vocab))))\n",
    "    unknown_words = sorted(list(set(vocab).difference(set(pretrained_wv.key_to_index))))\n",
    "    ###########################################################\n",
    "    unknown_ids = [gensim_dict.token2id[w] for w in unknown_words if w not in special_tokens]\n",
    "    unknown_count = np.sum([gensim_dict.cfs[idx] for idx in unknown_ids])\n",
    "    cov = 1 - unknown_count / gensim_dict.num_pos\n",
    "    return cov"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "eYZTCStNPIpR",
    "outputId": "7d90894b-3688-4b41-ea54-758b7f436f2c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9982916388485253"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocab_coverage(dictionary, glove)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EKzOosVxPIpR"
   },
   "source": [
    "### Tokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "id": "Mf8fyy3mPIpS"
   },
   "outputs": [],
   "source": [
    "def make_vocab_from_wv(wv, folder=None, special_tokens=None):\n",
    "    if folder is not None:\n",
    "        if not os.path.exists(folder):\n",
    "            os.mkdir(folder)\n",
    "\n",
    "    # UPDATED\n",
    "    ###########################################################\n",
    "    # The index2word property isn't available anymore at the\n",
    "    # newer Gensim version. It was replaced by index_to_key\n",
    "    # words = wv.index2word\n",
    "    words = wv.index_to_key\n",
    "    ###########################################################\n",
    "    if special_tokens is not None:\n",
    "        to_add = []\n",
    "        for special_token in special_tokens:\n",
    "            if special_token not in words:\n",
    "                to_add.append(special_token)\n",
    "        words = to_add + words\n",
    "\n",
    "    with open(os.path.join(folder, 'vocab.txt'), 'w') as f:\n",
    "        for word in words:\n",
    "            f.write(f'{word}\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "id": "K4Y-ub7iPIpS"
   },
   "outputs": [],
   "source": [
    "make_vocab_from_wv(glove, 'glove_vocab/', special_tokens=['[PAD]', '[UNK]'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "id": "3NIjvZpFPIpS"
   },
   "outputs": [],
   "source": [
    "glove_tokenizer = BertTokenizer('glove_vocab/vocab.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IN_vm748PIpS",
    "outputId": "edcb30fd-df62-46dd-8aa3-d8109ad75ced"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[7101, 930, 2, 300, 12427]"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "glove_tokenizer.encode('alice followed the white rabbit', add_special_tokens=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "TwsVqJhtPIpS",
    "outputId": "eaf348f5-198d-42a8-9d3b-1649d0994c2f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(400002, 400000)"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(glove_tokenizer.vocab), len(glove.vectors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YBrqOhP-PIpT"
   },
   "source": [
    "### Special Tokens' Embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "id": "UzAqTQ9CPIpT"
   },
   "outputs": [],
   "source": [
    "special_embeddings = np.zeros((2, glove.vector_size))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4AWIyA3JPIpT",
    "outputId": "683c23aa-ed56-4178-e3aa-8f3ce9b8940d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(400002, 50)"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "extended_embeddings = np.concatenate([special_embeddings, glove.vectors], axis=0)\n",
    "extended_embeddings.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fMTUfdESPIpT",
    "outputId": "1e18391e-fb28-4937-dd70-0d8b3812ec1e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "alice_idx = glove_tokenizer.encode('alice', add_special_tokens=False)\n",
    "np.all(extended_embeddings[alice_idx] == glove['alice'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "k8_PGFaXPIpU"
   },
   "source": [
    "## Model I - GloVe + Classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rzpeWYcdPIpU"
   },
   "source": [
    "### Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "id": "QOc1ZhgdPIpU"
   },
   "outputs": [],
   "source": [
    "train_sentences = train_dataset['sentence']\n",
    "train_labels = train_dataset['labels']\n",
    "\n",
    "test_sentences = test_dataset['sentence']\n",
    "test_labels = test_dataset['labels']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "id": "r5SLBMioPIpU"
   },
   "outputs": [],
   "source": [
    "train_ids = glove_tokenizer(train_sentences,\n",
    "                            truncation=True,\n",
    "                            padding=True,\n",
    "                            max_length=60,\n",
    "                            add_special_tokens=False,\n",
    "                            return_tensors='pt')['input_ids']\n",
    "train_labels = torch.as_tensor(train_labels).float().view(-1, 1)\n",
    "\n",
    "test_ids = glove_tokenizer(test_sentences,\n",
    "                           truncation=True,\n",
    "                           padding=True,\n",
    "                           max_length=60,\n",
    "                           add_special_tokens=False,\n",
    "                           return_tensors='pt')['input_ids']\n",
    "test_labels = torch.as_tensor(test_labels).float().view(-1, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "id": "3Gy9RIDEPIpU"
   },
   "outputs": [],
   "source": [
    "train_tensor_dataset = TensorDataset(train_ids, train_labels)\n",
    "generator = torch.Generator()\n",
    "train_loader = DataLoader(train_tensor_dataset, batch_size=32, shuffle=True, generator=generator)\n",
    "test_tensor_dataset = TensorDataset(test_ids, test_labels)\n",
    "test_loader = DataLoader(test_tensor_dataset, batch_size=32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j-3CdOpWPIpU"
   },
   "source": [
    "### Pre-Trained PyTorch Embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "id": "bGxrKlyDPIpV"
   },
   "outputs": [],
   "source": [
    "extended_embeddings = torch.as_tensor(extended_embeddings).float()\n",
    "torch_embeddings = nn.Embedding.from_pretrained(extended_embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "SpdJmQNsPIpV",
    "outputId": "8747d9e7-17ed-412e-ea08-0fda20996e79"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[   10,    14,   392,  ...,     0,     0,     0],\n",
       "        [  934,    43,    59,  ...,     0,     0,     0],\n",
       "        [   10,  3204,     3,  ...,     0,     0,     0],\n",
       "        ...,\n",
       "        [   43,  4286,    85,  ...,     0,     0,     0],\n",
       "        [   10,     2, 10827,  ...,     0,     0,     0],\n",
       "        [   18,  7101,     4,  ...,     0,     0,     0]])"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "token_ids, labels = next(iter(train_loader))\n",
    "token_ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rwyGjFNSPIpV",
    "outputId": "9e57396d-b8ed-498e-ddad-5302413cd36b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([32, 60, 50])"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "token_embeddings = torch_embeddings(token_ids)\n",
    "token_embeddings.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UIHGuQ4WPIpV",
    "outputId": "88bcba14-e2a5-43ad-c6b7-a5699fbadef1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.0523,  0.0270, -0.0385,  ..., -0.0222,  0.0700,  0.0370],\n",
       "        [ 0.0106,  0.1402,  0.0343,  ..., -0.0981, -0.0004,  0.1091],\n",
       "        [ 0.0230,  0.2380, -0.0286,  ..., -0.1479,  0.0906,  0.2278],\n",
       "        ...,\n",
       "        [ 0.0250,  0.0515,  0.0371,  ..., -0.0343, -0.0120,  0.0927],\n",
       "        [ 0.1871,  0.1755, -0.1458,  ..., -0.1637, -0.0141, -0.0173],\n",
       "        [ 0.0118,  0.0111, -0.0040,  ..., -0.0244,  0.0071,  0.0365]])"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "token_embeddings.mean(dim=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PT2eZ1EqPIpV",
    "outputId": "359ae0df-c96e-4784-e8d5-20e3d5441391"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.0523,  0.0270, -0.0385,  ..., -0.0222,  0.0700,  0.0370],\n",
       "        [ 0.0106,  0.1402,  0.0343,  ..., -0.0981, -0.0004,  0.1091],\n",
       "        [ 0.0230,  0.2380, -0.0286,  ..., -0.1479,  0.0906,  0.2278],\n",
       "        ...,\n",
       "        [ 0.0250,  0.0515,  0.0371,  ..., -0.0343, -0.0120,  0.0927],\n",
       "        [ 0.1871,  0.1755, -0.1458,  ..., -0.1637, -0.0141, -0.0173],\n",
       "        [ 0.0118,  0.0111, -0.0040,  ..., -0.0244,  0.0071,  0.0365]])"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "boe_mean = nn.EmbeddingBag.from_pretrained(extended_embeddings, mode='mean')\n",
    "boe_mean(token_ids)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UDyYigyhPIpV"
   },
   "source": [
    "### Model Configuration & Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "id": "3yaB00G4PIpW"
   },
   "outputs": [],
   "source": [
    "extended_embeddings = torch.as_tensor(extended_embeddings).float()\n",
    "boe_mean = nn.EmbeddingBag.from_pretrained(\n",
    "    extended_embeddings, mode='mean'\n",
    ")\n",
    "torch.manual_seed(41)\n",
    "model = nn.Sequential(\n",
    "    # Embeddings\n",
    "    boe_mean,\n",
    "    # Classifier\n",
    "    nn.Linear(boe_mean.embedding_dim, 128),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(128, 1)\n",
    ")\n",
    "loss_fn = nn.BCEWithLogitsLoss()\n",
    "optimizer = optim.Adam(model.parameters(), lr=0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "id": "KTWpbPdfPIpW"
   },
   "outputs": [],
   "source": [
    "sbs_emb = StepByStep(model, loss_fn, optimizer)\n",
    "sbs_emb.set_loaders(train_loader, test_loader)\n",
    "sbs_emb.train(20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 395
    },
    "id": "qiaVP9uvPIpW",
    "outputId": "7f13cf99-1117-4c91-e017-293cd3555407"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9EAAAF6CAYAAAAedNt3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACYaElEQVR4nOzdd1xV9f8H8Ne5mykqgoshLnJbOXHPFAduzbRMzVxpllmWZmlav68Nc1uWM1dpKs6cIGpm7ty4cOJC5t3n98cVFLnABS6Xc+H1fDx4AOece8778uZceN/PEuLi4kQQERERERERUbZkBR0AERERERERkbNgEU1ERERERERkIxbRRERERERERDZiEU1ERERERERkIxbRRERERERERDZiEU1ERERERERkIxbRRERERERERDZiEU1ERERERERkIxbRRERERERERDZiEU1ERERERERkIxbRRERERERERDZiEU2SodVqceXKFWi12oIOhZ5iTqSF+ZAe5kR6mBPpYU6khfmQHubE+bCIJkkxmUwFHQK9gDmRFuZDepgT6WFOpIc5kRbmQ3qYE+fCIpqIiIiIiIjIRiyiiYiIiIiIiGzEIpqIiIiIiIjIRiyiiYiIiIiIiGzEIpqIiIiIiIjIRiyiiYiIiIiIiGykKOgAiIiIiIjoGbPZjKSkpHxZN9hsNkOlUuHJkydISEiw+/kp55iT/KXRaODm5gaZzH7txyyiiYiIiIgkwmw24+HDh3B3d4e3tzcEQbD7+fV6PVQqlV2LCso95iT/iKIIrVaLhw8fomTJknb7+TJLJDkGAxAXV9BREBERETleUlIS3N3d4eLiYvcCmqioEQQBLi4ucHd3R1JSkt3OyyKaJCM+XsDy5b5o2NAbkye7FHQ4RERERA6n1Wqh0WgKOgyiQkWj0dh1eASLaCpw164J+OQTDerW9caPP/rh1i051qxR4v59vvtKRERERQ9boInsy973FItoKnCzZ6sxf74aSUnPfh11OgE//6wqwKiIiIiIiIgyYhFNBW74cD0EQcywffFiFVJSCiAgIiIiIiKiTLCIpgJXqZIZr71mzLD9wQMZ1q1TFkBERERERFRUeHl5ITQ0NE/niIyMhJeXF2bMmGGnqEjKuMQVScLIkTps25axYJ47V4033jCAs/0TERERFV5eXl45Oj6OS7lkq2bNmoiNjcW9e/cKOpRCh0U0SUJIiAm1ahlw6lT6QvrCBTl271agbduMLdVEREREVDhMmDAhw7b58+cjPj7e6j57OnLkCFxc8rYyzCuvvIIjR46gZMmSdoqKpIxFNEmCIADvvpuMESOKZdg3Z46aRTQRERFRIfbJJ59k2Pbbb78hPj7e6j57qlKlSp7P4erqapfzkHNgJ1mSjM6dtfDx0WfYvn+/AqdP81eViIiIqKi7fv06vLy8MHz4cFy4cAH9+/dHhQoV4OXlhevXrwMANm/ejMGDB6Nu3booU6YM/P390aFDB2zcuNHqOa2NiR4+fDi8vLxw7do1LFiwAPXq1YOPjw9q1KiBr7/+GmazOd3xmY2JrlmzJmrWrInExERMmDABwcHB8PHxQePGjTON5/r16xg0aBACAwNRrlw5dOzYEVFRUZgxYwa8vLwQGRmZ2x9fppKSkjB9+nTUq1cPvr6+CAwMRO/evXH48OEMx2q1WsyePRshISHw9/dH2bJlUbNmTbz11ls4ffp02nFmsxnLli1Dq1atEBgYiNKlS6NatWro06dPvjwHR2JLNEmGUgn07XsPP/7ol2Hf3LlqLFjAqbqJiIiICLh69Sratm2LatWq4fXXX8ejR4+gUlmWR/3yyy+hVCrRsGFDlC5dGg8ePMC2bdvw5ptv4ptvvsGwYcNsvs7kyZMRFRWF9u3bo1WrVtiyZQu+/vprGAwGTJo0yaZzGI1GdO/eHXFxcejcuTNSUlKwfv16vPXWW/jjjz/QokWLtGNv376N9u3b4+7du2jTpg1q1aqFS5cuoVu3bmjWrFmOfka20mq16NKlC/7991/Url0bw4cPR2xsLDZs2IDdu3dj8eLFCAsLSzt++PDh2LBhA6pXr47XX38darUat27dQmRkJI4fP46aNWsCAL744gvMmjULFSpUQK9eveDu7o7bt2/j8OHD2L9/P5o2bZovz8cRWESTpISFPcDixeXSrRkNAH/8ocTnn2tRpkzGpbCIiIiIioq2bd3y9HhRBETRDEGQQRDsFNRTf/2VZN8TZuHw4cP46KOPMHHixAz71q1bh8DAwHTbEhMT0a5dO3z11VcYMGAAXF1dbbrOyZMnERUVhdKlSwMAPvroI7z88stYtGgRJkyYkFa4Z+XOnTuoW7cuwsPD047v1asXunbtirlz56YroqdMmYK7d+9i0qRJ+OCDD9K2L1++HKNHj7Yp5pyaNWsW/v33X/Tu3RsLFy6E8PQXY9iwYWjbti3GjBmD1q1bw8PDA0+ePMGff/6JOnXqYPfu3ZDL5WnnMZlMSEhISPt+2bJlKFOmDKKiojL8vB8/fpwvz8VRWESTpHh4mPD66yn46af0fyAMBgE//aTC5Mm6AoqMiIiIqOD98w//fQcAX19ffPjhh1b3vVhAA4C7uztef/11fPbZZzh27BiaNGli03XGjx+fVkADQMmSJdGxY0esWrUKly5dQvXq1W06z/Tp09MV3M2bN4efnx+OHTuWtk2n02Hjxo0oVaoURo0ale7xb7zxBn788UdcunTJpuvlxKpVq6BUKvH555+nFdAAULt2bfTr1w9Lly7Fli1b0LdvXwiCAFEUodFoIHth+Ry5XJ5hlnWlUpmu0E5VvHhxuz8PR+JAU5KcIUOSIZNlbHH+5RcVkhz3BicRERERSVSNGjUybQW+f/8+Jk6ciPr166NMmTLw8vKCl5cXPvvsMwDA3bt3bb5OnTp1MmwrV64cAODJkyc2naNYsWJWC/ty5cqlO8fly5eh0+lQt25dqNXqdMcKgoD69evbHLet4uPjce3aNQQFBaU9r+eldrlOHevs6emJdu3a4fDhw2jWrBm+/fZb/P333zAYDBke26NHD9y4cQONGjXCtGnTsH//fqSkFI7hmSyiSXICAszo3DnjbNxxcTKsWpV9lxkiIiIiKtxKlSpldfvjx4/RsmVLzJs3DyVKlMAbb7yBDz/8EBMmTEDHjh0BWFp8beXh4ZFhW2rLqslksukcnp6eVrfL5fJ0E5SldoX29va2eryPj49N18uJ1Gtm9vP09fVNdxwALFmyBB9++CHi4+MxdepUtG/fHhUrVsTHH3+M5OTktOO+/vprTJ06FSqVCjNnzkTXrl1RoUIFvPvuu3j48KHdn4sjsYgmSRo50vqL27x5Ktj4ekVEREREhZSQyYDu5cuX4+bNm/j000+xfft2/O9//8Nnn32GTz75BPXq1XNwlDmTWrA/ePDA6v7Y2Nh8u+b9+/ezvObzbya4urris88+w8mTJ3HixAnMnj0blSpVwoIFC9KNUVcoFBg9ejQOHz6Mc+fO4eeff0ajRo2wevVqDB061O7PxZE4qIIkqX59E+rVM2YY93PlihzbtysQGsp1o4mIiKjoqVcvb/8D5efEYlJw9epVAEhrdX7eoUOHHB1OjlSqVAlqtRonTpyATqdL16VbFEX8888/dr+mp6cnAgMDceXKFdy+fRtly5ZNt//AgQMAkDbj9osCAwMRGBiInj17onLlyti2bRt++OGHDMeVKVMGPXv2RPfu3fHqq69i3759SElJgYuLi92fkyOwiCbJGjVKhzffzPgrOmeOmkU0ERERFUl5nQHbbDZDr9dDpVJlmBiqMPDzsyyVevjw4XSTfq1btw47d+4sqLBsolar0bVrV6xduxbz58/H2LFj0/atWrUKFy9ezJfr9uvXDzNmzMAXX3yBBQsWpLXynzlzBr/99hs8PT3T1tF+8OABYmNjUa1atXTniIuLg06nQ4kSJQBYusyfOHECDRo0SHdcUlISkpKSoFQqnfr3j0U0SVZoqBH+/mbcuJH+Bjt0SIFjx+R4+WX26yYiIiKiZ/r06YMffvgBH330ESIjI+Hn54czZ85g//796Ny5MzZv3lzQIWZp8uTJ2LdvH6ZMmYKoqKi0daJ37NiBNm3aYNeuXTkqPg0GA4YPH57p/vnz52PMmDHYuXMn1qxZg4sXL6J58+a4f/8+NmzYAKPRiIULF6Z15759+zaaNWuGGjVqoHr16ihbtiwePXqErVu3wmAwpC3DlZKSgvbt26NSpUqoU6cOypcvj6SkJGzfvh337t3D6NGjM0ye5kxYRJNkKRTA8OE6fPJJxm4ec+eqsHhx4Zjdj4iIiIjso1y5ctiyZQs+//xz7Nu3DyaTCbVq1cKGDRtw8+ZNyRfR5cuXx86dOzFlyhTs2bMHUVFRqF27NtavX48///wTgPXJzjJjNpuxatWqTPfPnz8fGo0GmzZtwg8//IANGzZg3rx5cHFxQUhICMaNG4dGjRqlHe/v74+PP/4YERER2L9/Px49eoSSJUuidu3aePfdd9GmTRsAgJubG7744gvs378fhw4dwv379+Hl5YVKlSrh888/R48ePXL3A5IIIS4uLuNaQkQFQKvVIiYmBn5+ftBoNACAhASgenVPxMenH7Qjl4s4cSIBfn789c1P1nJCBYf5kB7mRHqYE+lhTnLm/v37mc6UbA+FvTu3M7I1J6+99hqOHDmCGzduwN3d3YERFg72vLd455CkeXgAb72lz7DdZBKwaJHzdgEhIiIiIrLG2jrWa9asweHDh9GiRQsW0BLA7twkee+8o8PcuSqYTOlbo5cuVWH8eC0yWXqPiIiIiMjpNGrUCLVq1ULVqlUhl8tx+vRpHDhwAB4eHpg6dWpBh0dgSzQ5gfLlRXTrZsiwPT5ewIoVqgKIiIiIiIgof7z99tu4f/8+Vq9ejZ9++gmXL19Gr169sHv37nQzjlPBYUs0OYWRI/X4/feMBfP8+Wq8844eCv4mExEREVEhMGnSJEyaNKmgw6AssCWanELduiY0bpxxbeiYGBnCw1lBExERERGRY7CIJqcxapTO6vY5c9QQOUk3ERERERE5AItochqvvWZExYqmDNuPHlXgyBF5AURERERERERFDYtochoyGTBiRMblrgBLazQREREREVF+YxFNTqVvXz2KFzdn2B4ersDVq/x1JiIiIiKi/MWqg5yKmxvw9tsZW6NFUcCCBVzuioiIiIiI8heLaHI6Q4fqoVRmnElsxQoV4uIcHw8RERERERUdLKLJ6ZQuLaJnT0OG7UlJApYuZWs0ERERERHlHxbR5JRGjrS+3NXChWrorc89RkRERERElGcsoskp1ahhRosWGVujb9+W4c8/lQUQERERERFJ2cqVK+Hl5YWVK1em216zZk3UrFkzz+expxkzZsDLywuRkZH5dg3KPRbR5LRGjcp8uSsx45BpIiIiIpKoIUOGwMvLC7///nuWx8XHx6NMmTLw9/dHSkqKg6Kzv8jISHh5eWHGjBkFHYpNUov6P/74o6BDkQQW0eS0Wrc2IjjYlGH7qVNyHDggL4CIiIiIiCg3BgwYAABYsWJFlsf98ccfSElJQc+ePeHi4mKXa2/atAmbNm2yy7ns5Z133sGRI0fwyiuvFHQoZAWLaHJaggCMGGF9bPScOWoHR0NEREREudWsWTMEBAQgIiICMTExmR6XWmSnFt32UKFCBVSoUMFu57OHkiVLokqVKnB1dS3oUMgKFtHk1Hr3NsDb25xh+44dSly6xF9vIiIiImcgCAL69+8Ps9mc6Vjjc+fO4d9//0X16tVRt25dPHnyBD/88AM6duyI4OBglCpVCsHBwRg2bBiuXr1q87UzGxP9+PFjvP/++6hcuTLKlCmDli1bYvPmzZmeZ/ny5ejXrx9q1qwJX19fBAYGonv37oiIiEh33IwZM9C5c2cAwDfffIMSJUqgdOnSKFGiBK5fv552TGZjordt24ZOnTrB398fpUuXRkhICObMmQOj0ZjuuOvXr8PLywvDhw/HlStX0L9/fwQEBKBs2bLo2rUrTp8+bfPPKKdsjREAIiIi0LNnTwQHB8PHxweVK1dGhw4dsGTJknTHnThxAgMHDkSNGjXg4+ODihUromXLlpg5c2a+PY/MKBx+RSI70miAIUP0+PprTYZ98+er8N132gKIioiIiCh/uLVtm7cTiCLMogiZIFi69dlR0l9/5enxr7/+Or7++mv89ttvmDBhAoQX4kstrlNboS9evIjp06ejadOm6NSpE1xdXXHx4kX8/vvv2LlzJ/bv3w9/f/9cxZKcnIzQ0FCcPXsW9evXR0hICG7duoW3334brVq1svqY8ePHo0aNGmjRogW8vb1x+/ZtbN26FWFhYVi+fDlCQ0MBAE2aNMGNGzewatUqhISEICQkBCaTCXK5HMWKFcsyrjlz5uCzzz5D8eLF0bNnT7i6umLbtm347LPPcOjQIaxYsSLDz+3GjRto06YNgoOD8cYbb+Dq1avYunUrOnfujCNHjsDHxydXPyN7xLhjxw707dsXxYoVQ8eOHVG6dGk8ePAAZ86cwZo1a/DWW28BAE6dOoX27dtDLpejY8eO8PPzw5MnT3D+/HksWbIEH374oV2fQ3ZYRJPTGzJEj++/V0OnS/+C8dtvKnz6qQ4lS3KWMSIiIiocFP/8U9Ah5Jvy5cujVatW2LVrFyIiItC8efO0fUajEWvXroVarUafPn0AAFWqVMGFCxdQvHjxdOeJiIhAWFgYZs6ciR9//DFXscyaNQtnz57Fm2++iVmzZqVt79OnD3r06GH1MYcPH0ZgYGC6bXfv3kXLli0xefLktCK6adOmAIBVq1ahSZMmmDBhAvR6PVQqFWSyzHtSXr16FVOmTEGpUqWwd+9elC9fHgAwadIkhIWFYcuWLVizZg369u2b7nFRUVGYMmUKxo4dm7Zt2rRpmDlzJlauXIn333/f5p9LdnIa44oVKyCKIjZv3pyhN8CjR4/Svl6zZg10Oh1WrlyZ9nO0dpyjsL8rOT1vbxF9+2Zc7kqrFfDLL6oCiIiIiIiIciOzCca2b9+O2NhYdOzYMa1oLlasWIYCGrCMrw4ODsa+fftyHcfq1auhUqkwceLEdNtbt26drrh/3osFNACULl0anTt3RnR0NG7cuJHreABg3bp1MBqNGDVqVFpxCgBqtRpTpkwBAPz2228ZHhcQEID33nsv3bbUn/OxY8fyFJO9YrQ2SVyJEiVyfVx+YxFNhUJmE4z99JMKWvboJiIiInIKHTt2hLe3N8LDw/HkyZO07ZlNKBYZGYnXX38dVatWhbe3N7y8vODl5YWzZ8/i7t27uYohPj4e169fR1BQEHx9fTPsb9SokdXHXbt2De+99x7q1KkDX1/ftFgWLVoEALmOJ9WpU6cAWLqDv6h+/frQaDRWxznXrFkzQwt3uXLlACDdz9gechpjaqt+mzZtMH78eGzevBkPHz7M8Nhu3bpBJpPhjTfewMiRI/H777/j9u3bdo09J1hEU6FQtaoZ7dplbI2OjZXh99+VBRAREREREeWUUqlEnz59kJKSkrZm9L1797Br1y6UL18eLVq0SDv2zz//RJcuXRAZGYmGDRti+PDh+OijjzBhwgT4+flBr9fnKoaEhAQAgLe3t9X91sYQX7lyBS1atMDKlSsRGBiIQYMGYfz48ZgwYQJCQkIAADqd9UafnMZVqlSpDPsEQUCpUqXSjnmeh4dHhm0KhWVUr8mUcblYR8YYFhaGlStXolq1avjll18wYMAAVKpUCV26dEkryAHg1VdfRXh4OBo3bozff/8dQ4YMQbVq1dCqVasME7c5AsdEU6ExcqQOO3dmLJjnzVOjf3+DvefOICIiInI4Y716eTtBPk4sZi8DBgzA3LlzsXz5cgwePBhr1qyB0WhE//7907Wofv3119BoNNi3bx8qVqyY7hzr16/P9fVTi84HDx5Y3R8bG5th27x58xAXF4eFCxemjdlO9f777yMqKirX8bwY1/379zNMmCaKIu7fv2+1YHak3MQYGhqK0NBQJCQk4O+//8bmzZuxfPly9OzZE0eOHIGXlxcAoHHjxmjcuDFSUlJw9OhRbN++HYsXL0afPn1w6NAhq93p8wuLaCo0mjUzoUYNE86ckafbfvasHHv3KtCqVcYp9YmIiIicSV5nwDabzTZNYlWQgoODUa9ePfzzzz84c+YMVq5cmbYE1vOuXr2K4ODgDAX03bt3ce3atVxf39PTEwEBAbhy5Qru3buXoUv3oUOHMjwmdUmtjh07ptsuiiL+/vvvDMfL5Zb/V3PSElyrVi2Eh4fjwIEDeOWVV9LtO3r0KLRaLerXr2/z+fJDXmL08PBAmzZt0KZNG5hMJqxYsQL//vsvWrdune44FxcXNG3aFE2bNkWxYsUwffp07N27F4MGDcq35/Uiad45RLkgCJbWaGvmzuUEY0RERETOInXs84cffogLFy6gRYsWGVo2/fz8cPXq1XQtw1qtFuPGjYPBkHGYX0706dMHer0e06dPT7d9z5492L9/f4bj/fz8AFhm6H7e999/j7Nnz2Y4PnVCtFu3btkcU69evaBQKDB37lzcuXMnbbter8fnn38OwLJMWEHKaYxRUVFW30i4f/8+AMuEZABw5MgRaK1MdPTicY7ClmgqVHr0MOCLL8y4ezf9+0O7dytx9qwM1aqZCygyIiIiIrJVt27d8Mknn6QVpS9OKAYA77zzDj766CM0a9YMXbp0gclkwt69eyGKImrUqIEzZ87k+vpjxoxBeHg4li5divPnz6Nx48a4desWNmzYgPbt22PHjh3pjh80aBBWrlyJgQMHIiwsDCVKlMDRo0dx8uRJq8dXqVIFZcqUwfr166FSqeDr6wuFQoFhw4ZlulZ0hQoVMGXKFHz22WcICQlBt27d4Orqiu3bt+PSpUvo2LFjhq7k9rZ48WLs2rXL6r6BAweiUaNGOYpxwoQJuHv3Lho2bAh/f38IgoDDhw/j33//Rb169dImcfvhhx9w4MABNGrUCAEBAdBoNDh58iT279+PwMBAdOrUKV+f94tYRFOholIB77yjx5dfajLsmzdPjTlzUgogKiIiIiLKCQ8Pj7RJp4oXL55hbWAAGDp0KJRKJRYtWoRly5ahWLFiaNeuHT7//HO8+eabebq+m5sbtmzZgi+++ALh4eE4efIkgoOD8csvvyA+Pj5DUVy7dm2sX78eX331FcLDwyGTydCgQQNs374d27Zty3C8XC7H8uXL8fnnn+OPP/5AYmIiAEsLeGZFNACMGjUKQUFBmDt3LtauXQu9Xo+KFSti2rRpePfddyHk8zj3gwcP4uDBg1b3NWnSBI0aNcpRjOPGjcPmzZtx4sQJ7NmzBwqFAv7+/vjiiy8wePDgtG7vgwcPhqenJ/79918cPHgQoiiifPny+OCDDzBixAh4enrm6/N+kRAXFyc69IpEmdBqtYiJiYGfnx80moxFsK0ePxZQvboHkpPTv4ioVCJOn06Ary9/5W1lr5yQfTAf0sOcSA9zIj3MSc7cv3/f6szG9uIMY6KLGubEMex5bzFLVOgULy6if/+MSxro9QJ+/pljo4mIiIiIKPdYRFOhNHy4HoKQscV58WIVkpMLICAiIiIiIioUWERToRQUZEbHjhmXtHr0SIY1a9gaTUREREREucMimgqtrJa7MnOSbiIiIiIiygUW0VRoNWpkwssvZ2yNvnxZjp07OTE9ERERERHlHItoKrQEARg5MuMEYwAwd65jF2QnIiIiIqLCgUU0FWpduxpQvnzGvtuRkQqcPMlffyIiIiIiyhlWEVSoKRTAsGGZjY1mazQREREREeUMi2gq9AYO1MPDI+NyV+vXK3HrllAAERERERFlThQz/t9CRLln73uKRTQVesWKAQMGZBwbbTQK+OknLndFRERE0qHRaKDVags6DKJCRavVQqPR2O18LKKpSBg2TAeZLOM7UL/+qkZiYgEERERERGSFm5sbEhMTkZKSwhZpojwSRREpKSlITEyEm5ub3c7LdX6oSAgIENG1qwEbNqRveX7yRMDKlSoMG2Z9Fm8iIiIiR5LJZChZsiSSkpLw4MEDu5/fbDantcrJZGxPkwLmJH9pNBqULFnSrj9bFtFUZIwcqc9QRAPA/PkqDBmih1xeAEERERERvUAmk8HDwwMeHh52P7dWq0V8fDx8fX3t2r2Vco85cT58q4OKjFdfNaFhQ2OG7deuybFlC99PIiIiIiKi7LGIpiJlxAjry13Nm8flroiIiIiIKHssoguBTZs2ISwsDIGBgfDy8sL169cLOiTJCg01IjDQlGH74cMKHD3K/txERERERJQ1FtGFQHJyMho3boyJEycWdCiSJ5cDw4dbn0Rs7lwud0VERERERFnjQNBCoG/fvgCAs2fPFnAkzqF/fz2mT9fgyRMh3faNG5W4fl2LgAAuJ0FERERERNY5bUv0Dz/8AC8vL3h5eeGff/5x+PXXrFmDsWPHokWLFvDx8YGXlxdWrlyZ5WOOHTuGXr16wd/fH2XLlkWbNm2wYcMGB0VMqdzdgUGDMo6NNpsFLFzIsdFERERERJQ5pyyiz549ixkzZth1weycmjZtGpYsWYKYmBj4+vpme3xERATat2+Pw4cPo1u3bhg0aBDu3buHQYMGYfbs2Q6ImJ43dKgeCkXGFufly1V48qQAAiIiIiIiIqfgdEW0wWDA8OHDUbNmTYSGhubosatXr8aNGzcy3W8ymTBnzhzo9dbHzD5v9uzZOHXqFKKjo/H2229neazRaMSYMWMgk8mwZcsWzJo1C1999RUOHDiASpUqYerUqRnimjJlSlpLe2YflHvlyono3t2QYXtCgoBlyzg2moiIiIiIrHO6InrmzJk4f/485syZA7nc9tmUb926hTFjxqBTp05WC2mz2Yzhw4fjs88+w5IlS7I9X4sWLeDv72/TtSMiInD16lX07NkTtWrVStterFgxjBs3Dnq9HqtWrUr3mFGjRuHIkSNZflDeZLbc1cKFahgy1tdERERERETONbHYiRMn8O2332LixIkIDg7O0WPLlSuHn3/+GYMGDULnzp0RHh4OPz8/AJYC+t1338XatWvRt29fDBkyxK5xHzhwAADQqlWrDPtat24NAIiKikq33dvbG97e3naNg9KrU8eMJk2MOHAg/W1w86YMmzYp0aMHK2kiIiIiIkrPaVqidTpdWjfuMWPG5OocnTt3xuLFi3Hr1i106tQJMTExaS3Qa9euRe/evTFv3jzIZPb9sURHRwMAKlasmGGfr68v3N3dceXKlVyf//Hjxzh16hQuXboEALhw4QJOnTqFx48f5/qcRcWoUdZbo+fMUUHkJN1ERERERPQCp2mJnj59OqKjo7Fv374cdeN+UdeuXWEymTB06FB07twZdevWxYYNG9CjRw/Mnz/f7gU0AMTHxwMAPD09re738PBIOyY3tm7dipEjR6Z937t3bwDA3Llz0b9//1yftyho186IypVNuHQp/e/U8eMKHDokR+PGpgKKjIiIiIiIpMgpiugjR45g9uzZ+Pjjj1GtWrU8n6979+4wGo145513cO3aNYSGhmLRokV5Ks4LUv/+/Vks55JMBowYocf777tk2Dd3rhqNGycXQFRERERERCRVku/ObTQaMXz4cFSvXh3vv/++Xc4piiIiIiLSvj9//jxiY2Ptcm5rUlugM2ttTkhIyLSVmvJfnz56lChhzrB961YFoqMlf4sQEREREZEDSb5CSExMRHR0NE6fPo1SpUqlW+IpdUbrtm3bwsvLC+Hh4dmeTxRFjBkzBitWrED37t2xaNEiXLt2DZ07d8adO3fy5TmkjoVOHRv9vHv37iExMRFBQUH5cm3KnqsrMHhwxmXNRFHAggVc7oqIiIiIiJ6RfHdutVqNAQMGWN138OBBREdHo0OHDvD29s52ySlRFDF27FgsW7YM3bp1w08//QS5XA6ZTIZhw4alzdpdunRpuz6HkJAQfPfdd9izZw969OiRbt/u3bvTjqGCM2SIHrNmqaHXC+m2r1ypwsSJOhQvzlnGiIiIiIjICYpoFxcXzJ492+q+4cOHIzo6GuPGjUO9evWyPI8oihg3bhyWLl2aroAGgJ49ewJAukLa19fXbs+hefPmCAwMxO+//45hw4alrRX95MkTfPfdd1CpVOjbt6/drkc55+srolcvA1auTN/ynJws4NdfVRg3zvos3kREREREVLRIvoi2lzt37iA8PBxhYWH46aefoFCkf+o9e/aEKIp49913sW/fPvTp0yfL8y1btgyHDh0CAJw9exYAsHz58rQ1oRs1aoSBAwcCABQKBX788Uf06NEDoaGh6N69O9zd3bFp0ybExMRg6tSpCAgIsPdTphwaOVKXoYgGgEWLVBg1SgcVe3YTERERERV5RaaILlu2LP766y+UL18+QwGdqlevXqhbty4qVaqU7fkOHTqUNiY71eHDh3H48OG071OLaABo1qwZtm/fjhkzZmDDhg0wGAyoVq0avvjiC3Tv3j2Xz4rsqVo1M1q1MmDPHmW67XfvyvDHH0r062cooMiIiIiIiEgqhLi4OA72JEnQarWIiYmBn58fNBpNgcSwZ48C3bu7Zdheo4YJkZGJEAQrDyrEpJATeob5kB7mRHqYE+lhTqSF+ZAe5sT5SH52biJHatnSiGrVTBm2nzkjR0SEc64jTkRERERE9sMimug5ggCMGGF9ErG5c9UOjoaIiIiIiKSGRTTRC3r1MsDHx5xh+86dSly4wFuGiIiIiKgoY0VA9AK12rJutDXz5rE1moiIiIioKGMRTWTF4MF6aDQZ59xbvVqJ+/eL2OxiRERERESUhkU0kRUlS4ro1y9ja7ROJ2DxYi4YTURERERUVLGIJsrEiBHWu3T//LMKWq2DgyEiIiIiIklgEU2UicqVzWjf3pBh+4MHMsycqYbIFdaJiIiIiIocFtFEWRg1yvpyVzNnavDJJxqYM07iTUREREREhRiLaKIsNGliQq1aJqv7FixQY/BgF+is19lERERERFQIsYgmyoIgAFOmZD4AesMGFXr2dMOTJw4MioiIiIiICgyLaKJstGplxP/+lwJBsD4IOjJSgdBQd9y9y6WviIiIiIgKOxbRRDYYOlSPJUuSoVJZL6TPnJGjbVt3XLrEW4qIiIiIqDDjf/xENura1Yj165Pg6Wm9kI6JkaF9ezf884/cwZEREREREZGjsIgmyoEmTUzYti0RZcpYn5b70SMZunRxw/btCgdHRkREREREjsAimiiHqlc3Y+fORFSpYn3W7pQUAf37u2L5cqWDIyMiIiIiovzGIpooF/z8RGzfnoT69Y1W95tMAkaPdsX//qeGaL33NxEREREROSEW0US5VKKEiD//TEKHDoZMj/nqKw0+/FADk/VGayIiIiIicjIsoonywNUVWL48GW+9pcv0mMWL1XjrLVdoM19umoiIiIiInASLaKI8UiiA77/X4uOPM6+SN29Wols3N8TFOS4uIiIiIiKyPxbRRHYgCMDHH+vwww/JkMmsD4I+dEiBDh3ccfOm4ODoiIiIiIjIXlhEE9nRW28ZsHx5MjQa64X0uXNytG/vjnPneOsRERERETkj/idPZGehoUb8+WcSvLysryV965YMr73mjkOH5A6OjIiIiIiI8opFNFE+aNjQhO3bk1C+vPVC+skTAd26uSE8XOHgyIiIiIiIKC9YRBPlk+BgM3buTES1atbXt9JqBQwc6IpfflE5ODIiIiIiIsotFtFE+ahsWRFbtyaicWOj1f1ms4Bx41wwfboaovVh1EREREREJCEsoonymZcXsH59Erp0MWR6zP/9nwZjxrjAaL3WJiIiIiIiiWARTeQAGg3w66/JGDpUl+kxy5ap8MYbrkhOdmBgRERERESUIyyiiRxELgf+7/+0mDRJm+kx27cr0bWrGx494lrSRERERERSxCKaJMdl6VIIt24VdBj5QhCADz7QYe7cZMjl1gdB//OPAu3bu+HGDRbSRERERERSwyKapMNsRvlZs+A5YQLcevYE4uIKOqJ807+/AatWJcPV1XohfemSHO3auePMGd6iRERERERSwv/QSRoMBniOGYPSK1YAAOTnzsGtb18gJaWAA8s/7doZsWlTEkqUsL6W9N27MnTs6I7ISLmDIyMiIiIiosywiKaCl5QE1/794bJuXbrNisOH4TpoEArzlNWvvmrCzp1J8Pe3XkjHxwvo0cMNf/6pcHBkRERERERkDYtoKngGA2Q3b1rdpdy+HS5jx6IwL6JcqZIZO3cmomZNk9X9er2AQYNcsXChysGRERERERHRi1hEU8Hz8kLS77/DVK6c1d2qFSugnjbNwUE5VunSIrZsSUTz5tZb3UVRwIQJLvjiC3Vhfj+BiIiIiEjyWESTJIhly+Lx6tUwFCtmdb/m22+hWrDAwVE5lqcnsG5dEnr21Gd6zPffazB8uAsMBgcGRkREREREafK9iI6Li8PZs2eh0+ny+1Lk5EyVK+PyDz9AdHGxul/zySdQ/vGHg6NyLJUKWLQoBSNGZH6/rF6tQr9+rkhMdGBgREREREQEwA5F9MmTJ/HVV19hz5496banpKRg8ODBCAoKQpMmTRAcHIyNGzfm9XJUyCXVqIG4xYshKjJOpCWIIlzefReKvXsLIDLHkcmA6dO1mDo185nJd+1SoksXNzx4wLWkiYiIiIgcKc9F9IoVK/Dtt99CfGGg5vTp07F+/XqIoghRFBEXF4ehQ4fi7Nmzeb0kFXL6Vq2QMneu1X2CwQDXAQMgP37cwVE53ujReixalAyFwvog6GPHFGjXzg3XrrGQJiIiIiJylDwX0QcPHoRGo0HLli3Ttun1eixduhRKpRJr167FtWvXMGzYMBgMBiwo5ONayT4MffogJZPJxITERLj26gVZdLSDo3K83r0NWLs2Ge7u1gvpK1fkaNfOHSdOcHoDIiIiIiJHyPN/3rGxsShTpgxksmenOnLkCBISEtChQwe0bdsWxYoVw+effw43NzdERUXl9ZJUROhHjYLuvfes7pM9eAC3bt0g3L3r4Kgcr1UrI8LDE1GqlPW1pGNjZejUyR379skdHBkRERERUdGT5yI6Li4OxYsXT7ftyJEjEAQBrVu3Ttvm4uKCwMBA3L59O6+XpCJEO2UK9H37Wt0nu3EDbj16AE+eODgqx6tTx4ydO5MQFGR9LenERAG9erlh3TqlgyMjIiIiIipa8lxEu7i44MGDB+m2HTp0CADQoEGDdNtVKlW6FmuibMlkSJk9G4Z27azulv/3H9xefx3Qah0cmONVqGDGjh1JqFvX+lrSBoOAoUNdMXu2CibrtTYREREREeVRnivaKlWq4MaNGzh37hwA4OHDh4iMjETJkiVRtWrVdMfeuXMH3t7eeb0kFTVKJZJ//RXGevWs7lZERcF1yBAUhcqxVCkRmzcnoXXrzBeKnjTJBbVre2D6dDWuX+ekY0RERERE9pTnIjosLAyiKKJXr1749NNP0blzZ+j1enTv3j3dcTExMbh79y6CgoLyekkqitzckLxmDUxVqljdrQwPh+aDDwDR+gRchYm7O7B6dTL69NFneszNmzL83/9pUKeOB8LCXPHHH8qi0FhPRERERJTv8lxEv/POO2jcuDFu3bqFefPm4dy5c6hUqRImTJiQ7rgNGzYAAJo2bZrXS1IRJZYogaQ//oC5XDmr+9VLlkD99dcOjqpgKJXAggUpGDs268pYFAXs26fE4MGuCA72wEcfaXDqFIdUEBERERHlliKvJ1CpVNi8eTO2bduGS5cuwc/PD6GhodBoNOmOk8vlePfdd9G1a9e8XpKKMNHPD0m//w63Dh0gi4vLsF/zzTcQfXygHzzY8cE5mCAAU6boULq0iE8+0UAUs+66HRcnw6JFaixapEbt2iYMGKBHz556eHk5Jl4iIiIiosIgz0U0AMhkMoSGhmZ5zMiRI+1xKSKYX3oJyWvWwC0sDEJKSob9mg8/hNnbG8Yi8obNu+/qERBgxvjxLrh507ZW5pMn5Th50gWffaZBly4G9O+vR9OmJnDePyIiIiKirPFfZnJKpgYNkPzLLxDlGddGFkQRrkOHQh4RUQCRFYwOHYw4cSIBq1cnoVMnAxQK28aGa7UC1q5VoWtXd7z8sjv+9z81bt3iZGRERERERJnJcxH94MED7N+/H5cvX86w79dff0VISAiCgoLQq1cvXLp0Ka+XI0pj7NABKbNmWd0n6PVw698fspMnHRxVwVEogNdeM2LFimScPZuAqVNTUKWK7TOWX7smx1dfaVCzpgd69XLFxo0K6DOfu4yIiIiIqEjKcxG9YMECdOvWDf/880+67UuWLMEHH3yAs2fP4vHjx9i1axc6d+6MR48e5fWSRGkMb7yBlClTrO4TEhLg1qsXZFevOjYoCfDxETF6tB5//52IHTsS8cYberi52dY6bTYL+OsvJd580w116pTC99+Xx/nzGVv8iYiIiIiKojwX0ZGRkZDL5ejcuXO67d999x0AYPTo0VixYgUaNWqE2NhYzJs3L6+XJEpHP2YMdMOHW90ni42Fa/fuEGJjHRyVNAgC0KCBCXPmpOD8+XjMnp2MBg2MNj/+0SMZfvutNFq08EabNm5YulSJ+Ph8DJiIiIiISOLyXETHxMTA19cX7u7uadtOnz6NmJgYNGjQAF9++SVCQ0Px66+/Qi6XY8eOHXm9JFF6ggDtV19B36uX1d3yq1fh1rMninr15+EBDBhgwI4dSfj77wSMHq1DqVJmmx9/9KgCY8a4IjjYE8OHu+DgQXlRWJabiIiIiCidPBfRjx49QunSpdNtO3z4MACgY8eOadt8fX0RFBSEa9eu5fWSRBnJZEiZOxeGVq2s7pafOgW3N94AdDoHByZNVauaMXWqFmfPJmDFiiS0b2+ATGZbRZycLGDVKhU6dnRHvXru+OEHFe7e5WRkRERERFQ05LmIlslkSExMTLftyJEjEAQBDRs2TLfd09MTes5URPlFpULysmUwvvyy1d2KiAi4DBsGmGyfbKuwUyqBTp2MWLMmGf/9l4DJk7UICrL953P5shxTprigenUP9O3rii1bFDAY8jFgIiIiIqICluci2t/fH1euXMHjx48BAAaDAXv27IGLiwvq1q2b7tiHDx+iZMmSeb0kUebc3ZG8di1MFSta3a36809oPv4Y7IecUZkyIsaN0+HffxOxZUsi+vbVw8XFtp+TySRg+3Yl+vd3Q/XqHvj8cw0uXeIKekRERERU+OT5v9xWrVrBYDBg8ODB2LZtG0aPHo1Hjx6hdevWUCgUacc9efIE165dQ7ly5fJ6SaIsid7eSFq/HuYXhhmkUv/0E9QzZzo4KuchCEBIiAkLFqTg5Mn7+OSTa6hTx/bm5dhYGWbNUqNePQ+89pobVqxQ4oXOKkRERERETivPRfTYsWPh6+uLvXv3on///lizZg00Gg0mTJiQ7rjt27dDFEU0atQor5ckypYYEICk33+H6Olpdb/mq6+gXLrUwVE5H09PEd27P8D27Y8QFZWAd9/VoUQJ2ycjO3xYgVGjLJORvfeeC/75h5OREREREZFzy3MR7ePjgz179mDYsGFo1aoV3nrrLezbtw/Vq1dPd9yhQ4dQo0YNtG/fPq+XJLKJuUYNJK1aBVGttrrf5f33oQgPd3BUzqt6dTO+/lqLc+cSsGRJElq3NkAQbKuIExMFLFumQtu2lsnIPvpIg/BwBeLi8jdmIiIiIiJ7E+Li4tguRJKg1WoRExMDPz8/aDQau51XER4O14EDIZgztqCKajWS1q+HKSTEbtcrTLLLSUyMgN9+U2HlShVu3Mj5e3KCIKJWLTOaNTOiWTMjGjUy4rnV8ugF+XWPUO4xJ9LDnEgPcyItzIf0MCfOhzP/UKFn7NQJKd9/b3WfoNPBrV8/yM6ccXBUhYOfn4gJE3Q4cSIBGzcmomdPPdRq29+XE0UBJ0/KMXu2Gr16uSEw0BPt27th2jQ1IiLk0GrzMXgiIiIiolxQZH+I7WJjY7F3715cunQJCQkJ8PDwQJUqVdCyZUuUKlXKnpciyhHDm29CGxsLzVdfZdgnxMfDrWdPJO7YATEgoACic34yGdC8uQnNm6fg8WMt1q1TYvlyFU6flufoPEajgL//VuDvvxWYORNQq0U0aGBKa6muW9cEpTKfngQRERERkQ3sUkTrdDpMmjQJS5cuhcHKIrFKpRKDBg3CF198AXUm41OJ8pvuww8hxMZC/dNPGfbJ7t6FW48eSNq+HaK3dwFEV3gULy7inXf0eOcdPU6ckGHFChXWrlUhPl7I8bl0OgEREQpERFheqtzdRTRubETTppaiumZNM2TsT0NEREREDpTnItpsNqNfv37Yt28fRFFEqVKlULlyZZQuXRp3797FpUuXcP/+fSxatAiXL1/GunXrIAg5/2eaKM8EAdqvv4Zw/z5Uf/6ZYbf88mW49uqFpM2bwYG59lGnjhl16mgxdaoW4eFKbN9uKYjv389d5ZuYKGDnTiV27rQ0RxcvbkaTJs9aqqtUMYMvL0RERESUn/JcRK9YsQJ79+6Fp6cnpk2bhn79+qVbH9pkMmHVqlWYNGkS9uzZg5UrV+KNN97I62WJckcuR8rChZA9egRFRESG3Yrjx+E6cCCSV68GVKoCCLBwcnEBevUyoFcvA0QROH9ehshIS0EdGanAkye5q3wfP5Zh82YZNm+2FNW+vua0VupmzYwICBBZVBMRERGRXeW5I+SaNWsgCAKWLVuGAQMGpCugAUAul+ONN97AkiVLIIoiVq1alddLEuWNWo2kFStgqlXL6m7lnj1wGTkSsDKbN+WdIAAvvWTGO+/osWJFMq5cice+fQmYOjUFbdsa4OaW+wUD7t2T4fffVXjvPVfUqeOJWrU8MHKkC9asUeL2bVbTRERERJR3eW6J/u+//xAQEIDmzZtneVzz5s0RGBiI//77L6+XJMo7T08k/f473Nq3h/zq1Qy7VevWQfT2hnb6dLApM3/J5andvvUYPVoPgwE4dkyeNhb6yBE5dLrc5SAmRoaVKy1LcAFA5crPun43aWJCyZJc4Y+IiIiIcibPRXRKSgqCgoJsOrZ48eK4c+dOXi9JZBeijw+S16+HW/v2kMXGZtivnj8fZl9f6MeOdXxwRZhSCTRoYEKDBiaMH69DSgpw5Ig8rfv3v//KYTLlrqi+dEmOS5fkWLzYMsFhjRqWorppUyMaNzaiWDF7PhMiIiIiKozyXET7+vri0qVLSElJgYuLS6bHJScn49KlS/Dx8cnrJYnsxlyhApLWrYN7p04QEhIy7HeZMgWitzcMHMdfYFxcUpfPMgHQISEBOHxYkdZSfeqUDKKYu6L6zBk5zpyRY948NWQyEXXrWorq9u2NqFfPBHnOVugiIiIioiIgz2OimzZtiqSkJEycODHL4yZOnIikpCQ0a9Ysr5cksitz7dpIWrECYiYTibmMGQPFtm0Ojooy4+EBtG1rxNSpWuzfn4grVxKwfHkShg7VITjYlOvzms0C/v1Xge+/1+C119xRtaplPPWWLQokJ9vxCRARERGRU8tzET1mzBgolUosXboUTZo0wYoVK3Ds2DHcvHkTx44dw4oVKxASEoJly5ZBpVLhvffes0fcRHZlat4cyT/9BNHK+GfBZILroEGQ//13AURG2SleXETnzkb8739aHD6ciAsX4vHzz8kYOFCPwMDcF9UPHljGU/fv74agIE/07euKZcuUiI3lGHkiIiKioizP3bmrVKmCBQsWYMSIEfjvv/+sFsmiKEKj0WD+/PmoUqVKXi9JlC+MXbtCO3MmXD74IMM+QauFa58+SNq2DeaXXiqA6MhWvr4ievY0oGdPAwDg+nUh3XJad+7k/L1DrVbA9u1KbN+uhCCIqFfPhI4dDejQgWtTExERERU1eW6JBoBu3bohIiIC/fv3h4+PD0RRTPvw8fHBgAEDEBERgbCwMHtcjijf6AcPhvajj6zuk8XFwa1HDwgxMQ6OivIiIEDEG28YsGhRCs6eTcDRown47rsUhIXpUbJkzpcxE0UBR44oMGWKCxo08MCrr7pj0iQNDh6Uw5T7hm8iIiIichJ5bolOVblyZcyZMwcAEB8fj8TERLi7u8PT0zPtmObNm+PJkyc4ceKEvS5LADZt2oRffvkFJ06cQFxcHE6ePImAgICCDstp6T75BEJsLNRLlmTYJ7t9G25hYUiZOxemhg0dHxzliSAAlSqZUamSHm+/bVkK/OxZWdokZVFRCiQk5KxZOTpajtmz5Zg9W42SJc1o396IDh0MaNXKCDe3fHoiRERERFRg7NIS/SJPT0+ULVs2XQENADdv3sSNGzfy45JFWnJyMho3bpzt5G5kI0GA9ttvYejUyepueXQ03F97Da5vvgnh2jXHxkZ2JZMBNWqYMWKEHqtXJ+Py5XisX5+EIUN0KFcu563UDx/K8NtvKgwY4IaKFT3Rp48rli5V4t499vcmIiIiKizs1hJNBadv374AgLNnzxZwJIWIXI7kn3+GW/fuUBw8aPUQ5caNUGzbBv2wYdB+8AHg5eXYGMnu1GqgVSsjWrWyTFR28qQMW7cqsXWrEmfO5Gy9K61WwI4dSuzYYRlH/eqrJnToYETHjgZUrcpx1ERERETOKl9aou1Nq9Vi4sSJ6NChA4KDg+Hr64sqVaqgffv2WLFiBQwGg8NjWrNmDcaOHYsWLVrAx8cHXl5eWLlyZZaPOXbsGHr16gV/f3+ULVsWbdq0wYYNGxwUMeWYRoOk336DqXr1TA8R9HqoZ8+Gx8svQ7VoEVAAv4uUPwQBqFPHjIkTdThwIBGnTsXjm29S0KKFAQqFmKNziaKAf/5R4MsvNWjY0AOvvOKOTz/VICpKDqMxn54AEREREeULpyiik5KS8Msvv0AQBLRr1w4jR45Ep06dcPv2bYwaNQp9+vSB2Zzzrpd5MW3aNCxZsgQxMTHw9fXN9viIiAi0b98ehw8fRrdu3TBo0CDcu3cPgwYNwuzZsx0QMeWKlxeS/vgjy0IaAGSPHsHlo4/g3rgxFNu3A2LOiiySPn9/EcOG6fHnn5Zu3z//nIwePfTw9Mx5rq9ckWPuXDVCQ91RpYoH3n3XBZs2KZCYmA+BExEREZFdOUV37uLFi+PGjRtQqVTpthuNRoSFhWHPnj3466+/0L59+yzPs3r1ajRu3Bj+/v5W95tMJsyfPx/vvPNOhmu9aPbs2QgKCoK/vz++//57fPHFF5keazQaMWbMGMhkMmzZsgW1atUCAHz00Udo3bo1pk6diq5du6aLa8qUKfjhhx+yjCEuLi7L/WQfYunSSNy9G6qFC6H59lsI8fGZHiu/dAluffvC2KwZUqZNg/lprqlw8fJC2jJaen0KoqIU2LpVgW3blLh5M2fvTT56JMPq1SqsXq2CWi2ieXNLl+/XXjOidGm+GUNEREQkNU7REi2TyawWtQqFAp2eTv505cqVLM9x69YtjBkzBp06dbI6uZnZbMbw4cPx2WefYYmVWZlf1KJFi0yL8RdFRETg6tWr6NmzZ1oBDQDFihXDuHHjoNfrsWrVqnSPGTVqFI4cOZLlBzmQRgP9mDFI+Pdf6AYPhijPenysIiIC7s2bw2XUKAh37jgoSCoIKhXQsqVlDPXp0wnYvz8BH3+sRa1aOV/vSqcTsHOnEmPHuiI42BOtW7vh22/VOHdOxs4NRERERBLhFC3RmTGbzdi9ezcAoFq1alkeW65cOfz8888YNGgQOnfujPDwcPj5+aWd591338XatWvRt29fDBkyxK5xHjhwAADQqlWrDPtat24NAIiKikq33dvbG97e3naNg/JOLFUK2m+/hX7oUGgmT4Zy585MjxVEEaoVK6Bcvx66MWOgGzUKXPOocBMEoHZtM2rX1uHjj3WIiRGwfbsSW7cqcOCAAgZDzmYT+/dfBf79V4GpUzWoUMGEdu0UqFvXA66uAsqWBScnIyIiIioAOS6iv/nmm1xfLCUlJdePBQC9Xo9vv/0Woiji8ePH2L9/Py5evIj+/fujefPm2T6+c+fOWLx4MQYPHoxOnTohPDwc5cqVw/Dhw7F27Vr07t0b8+bNg0xm3wb66OhoAEDFihUz7PP19YW7u3u2LelZefz4MWJiYnD16lUAwIULF/DkyRP4+fmhePHiuT4vZc4cHIzktWuh2LsXmk8/hTyLmdGF5GRoZsyAaulSaCdNgqFPH8vaSlTo+fmJGDpUj6FD9XjyBNi921JQ79ypRHx8zirgq1flWLjQDUBVAIBSKcLHR4Svrxk+PiJKlzbD11d8+mFO++zjI0KtzocnR0RERFRE5biI/vrrryHksvlDFMVcPxawFNHPF/GCIGD06NH4/PPPbT5H165dYTKZMHToUHTu3Bl169bFhg0b0KNHD8yfP9/uBTQAxD8dQ/viutmpPDw80o7Jja1bt2LkyJFp3/fu3RsAMHfuXPTv3z/X56XsGVu2RGJkJJQrV0IzbRpksbGZHiu7fRuuw4fDtGABUqZNg6lpUwdGSgWtWDGge3cDunc3wGBIwcGDcmzZYlk+K6fjqAHAYBBw65aAW7eyf2zx4tYL7Bc/FyvG1m0iIiKi7OS4iG7cuHGeCuG8cHd3R1xcHMxmM+7cuYPt27fjyy+/xD///IO1a9dmWqS+qHv37jAajXjnnXdw7do1hIaGYtGiRZBnM85Vqvr3789iuSDJ5TAMHAhDt25Qz5oF9Zw5ELTazA8/eRLunTvDEBoK7ZdfwmylhwIVbkol0Ly5Cc2bm/DNN1qcOfNsPeqTJ+3/OvT4sQyPHwPnz2d9nEbzrHXb19fSup3ayv38Zx8fEQqnHgxERERElHs5/jdoy5Yt+RFHjshkMpQrVw6DBw9GyZIl8dZbb+Hbb7/Ncobs54miiIiIiLTvz58/j9jYWJQpUyZf4k0t7jNrbU5ISICXl1e+XJscyMMDus8+g/6tt6D58kuo1q7N8nDlli1Q7NgB/ZAh0E2YAJFd74skQQBq1jSjZk0dJkzQ4dYtAdu2KbFtmwIRETkfR50XWq2AGzcE3LiRdeu2IIgoWVLMpNAW8dJLJlStamarNhERERVKTj8ws2XLlgCeTd6VHVEUMWbMGKxYsQLdu3fHokWLcO3aNXTu3Bl38mkW5dSx0Kljo5937949JCYmIigoKF+uTY4nli+PlEWLkLhnD4yNGmV5rGA0Qr1gAdzr1oVq3jxAr3dQlCRV5cqJGDJEjz/+SEZ0dDx+/TUZvXvrUayYdKbnFkUBDx7I8N9/cuzercSqVSp8/70Gn3zigkGDXNGwoQfq1PHAxx9rEBEhh8FQ0BETERER2Y/TF9F3794FACiVymyPFUURY8eOxbJly9CtWzf89NNP6N27NxYuXIirV6+ic+fOaeezp5CQEADAnj17MuxLnV089RgqPEwvv4ykrVuRtGwZTBUqZHmsLC4OLhMnwr1hQyg2bwbXMyIA8PQEunUzYNGiFFy+HI/16x9hwoTreP/9RLz5ph7t2xtQt64RZcuaoVBI63fm+nUZFixQo0sXd1Sq5ImhQ12wYYMSeZj+gYiIiEgSnGJU2/nz5+Hv7w9XV9d025OTk/Hpp58CANq2bZvlOURRxLhx47B06dK0Ajp1DHTPnj0BAMOGDUtb/srX19du8Tdv3hyBgYH4/fffMWzYsLS1op88eYLvvvsOKpUKffv2tdv1SEIEAcYuXZDYvj1UP/0Ezf/+B+HJk0wPl1+5ArcBA2Bs3Bgp06fDXKeO42IlSVMqgcaNDfDzuw8/Pw00mvTrUJvNwKNHAu7dE3Dvnuzp5+e/tnyOjZXleGbwvHryRMC6dSqsW6eCUimiSRMjOnY04rXXDPDzk1bxT0RERJQdpyiiN2zYgHnz5qFhw4bw9/eHh4cHbt++jV27duHRo0do1KgRRowYkeU57ty5g/DwcISFheGnn36C4oVZcXr27AlRFPHuu+9i37596NOnT5bnW7ZsGQ4dOgQAOPt0eaPly5endStv1KgRBg4cCABQKBT48ccf0aNHD4SGhqJ79+5wd3fHpk2bEBMTg6lTpyIgICBXPxtyEmo19KNGwfD661B/8w1UixdDMBozPVxx8CA8WrSAvk8faCdPhliunAODJWckkwHe3iK8vUVUr27O8tikJCA2NvNCO/Xz/fsCzGb7FtwGg4C9e5XYu1eJ8eNdUKuWCR06GNChgwG1a3McNREREUmfEBcXJ/lmgOPHj2PJkiU4cuQIbt++jaSkJHh6eqJ69ero0aMH3njjjQxFsTXXrl1D+fLlszz28uXLqFSpUrbnGj58OFatWpXp/n79+mH+/Pnptv3777+YMWMGjhw5AoPBgGrVqmHkyJHo3r17ttcrCrRaLWJiYuDn5weNRlPQ4eQr2eXL0EyeDOXWrdkeK7q4QDdqFHRjxgDu7g6I7pmilBNn4Oh8mEzAgwdZF9qpXycn5736LVfO/LSgNqJJE6NTrG/Ne0R6mBPpYU6khfmQHubE+ThFEU1FQ1F8AZFHRsLl008hP3Uq22PNvr7QfvopDP37Aw5ajq0o5kTKpJoPUQQSE4F792Q4fVqObdsU2LlTgbi43E+74eEhonVrS0Hdrp0RxYtL80+VVHNSlDEn0sOcSAvzIT3MifNx+onFiJyZqWlTJO7bh+R582DOZok12b17cH3vPbg3awb5vn0OiY/IFoIAeHgAlSqZ0yZCu3QpAZs3J2LECB0CA03Zn+QFCQkC/vxThWHDXFGpkgc6dXLD3LkqXLvG/t5ERERUsFhEExU0mQyG119HwtGj0H78McQXJtB7kfy//+AeFgbXPn0gu3DBQUES5YxSCTRtasL06VocP56IQ4cSMHmyFq++mvlcAJkxmQQcOKDAp5+6oE4dTzRq5I4vv1Tj6FE5zFkP/yYiIiKyOxbRRFLh5gbdxx8j4d9/oe/fH2I2Mywpd+yAe+PG0IwfD+HBAwcFSZRzggC89JIZ48bpsGtXEi5ciMePPyajfXsDNJqcd9M+d06O777ToE0bd7z0kgfGjHHB9u0KpKTkQ/CURqsFIiPlWLlSiWPH5FyJj4iIiiwW0UQSI5Ypg5S5c5G4fz+MTZtmeaxgMkH900/wePllqH78EdDpHBQlUe75+ooYONCANWuSER0dj5Urk9C/vx7e3jlvVr53T4alS1Xo29cNQUGeeP11V6xYocT9++z2nVd6PXDokBz/939qdO7shoAAT3Tu7I6RI13RqpU7+vRxRUwMf85ERFT0OMUSV0RFkblWLSRt2gTF9u3QTJoE+eXLmR4rxMfDZfJkqBcvhvbjj2EIDQU8PR0YLVHuuLkBoaFGhIYaYTIBR4/KsXWrEtu2KXDxYs4m0EtJEbB1qxJbtyohCCIaNDClzfZdpQr7fWfHaAROnJAjMlKByEg5Dh9WZDnr+s6dSjRsqMDkyVoMGaJ31HyHREREBY6zc5NkcGbCLBgMUP3yC9Rffw3Z48fZHi4qlTA2bw5Dp04wdugA0dc3V5dlTqSlqOXj8mUZtm1TYOtWJf7+W56nNasrVTKhQwcjOnQwoEEDk90KPmfOickEnD4tQ2SkAgcOKHDwoAIJCbn7Gdevb8SsWSl46aWCf7PCmXNSWDEn0sJ8SA9z4nxYRJNk8AXEBnFx0MycCdXChRAMBpseIgoCTPXrwxAaCmNoKMwVK9p8OeZEWopyPh4+FLBjhwLbtimxZ48CSUm5L6iLFRNRvboJ1aqZEBxsRnCwCS+9ZEbJkjn/c+hMOTGbgXPnZE9bmhWIipLnaRmyFymVIsaN02HcOF2BrvHtTDkpKpgTaWE+pIc5cT4sokky+AJiO9nVq9BMmQLlxo05fqzppZdgCA2FoVMnmGvXtsz6lAnmRFqYDwutFoiIUGDbNktRffeufQpBHx9zWlH9fIHt5ZVVLNLNiSgCly5ZiuaICAUOHJDj4cP8nwolONiEH39MQf36OV/azB6knJOiijmRFuZDepgT58Mx0UROyFyhApKXLoX80CFoPv0UimPHbH6s/Nw5yM+dg2bmTJjLlYOhY0cYOnWCqXFjy7pERBKn0QDt2hnRrp0R336rxYkTcmzdaimo//sv9/20Y2NliI2VISIi/Z/GMmXMeOkl03MFthlVq5rg4ZHXZ2JfoghcuyZDZKT8adGssNsbDGq1iPr1TWja1IgbN2RYsUKV6bHnz8vRvr0bhgzRY/JkreR+TkRERHnFlmiSDL4Ll0tmM5S//w71zJmQX7yY+9N4ecHYvr1lHHWrVoCbG3MiMcxH9q5dE7B9uxLbtikRFSWH0Zh/s0eXL29GlSoGlCv3GK++6oJateSoUsUEN7d8u2QGN24Iad2zDxxQ4OZN+xTNSqWIV1+1FM1NmxpRr54Jz//K7d8vx5gxLrh2Les3LcqXN+O771LQrl3O1wfPLd4n0sOcSAvzIT3MifNhEU2SwReQPBJFyM6fh3LLFijCw6E4cSL3p3JxgbFlSyS3a4cr1aujbM2azIkE8B7Jmbg4YNcuJbZuVWDXLiXi4/N/OSZBEBEQYOkW/tJLlrHWwcEmVKlihj1SdufOs6I5MlKebRFrK7lcxMsvpxbNJtSvb8z2zYDkZODrrzWYM0eV7aRvPXvq8fXXWnh75/+/HLxPpIc5kRbmQ3qYE+fDIpokgy8g9iXcvAnl1q1QhodDHhUFwZS78YmiTAZDw4Ywde4MQ2goRH9/O0dKtuI9knt6PXDwoBx79ypx9qwM587J7dZqawuZTESFCua0ovqllyxFdqVKZqgy7xmN+/cFHDhgKZgjIxW4dMk+RbMgiKhd24SmTS2Fc6NGxlx3uz5xQobRo11x+nTWsZUoYcb06Vr06WPIaiqGPON9Ij3MibQwH9LDnDgfFtEkGXwByT/C48dQbN9uaaXevRtCSkquz2WqVevZxGTVqmU5MRnZF+8R+4qPBy5ckOPcOUtRff685bO9xhHbQqEQUbHi8y3XJggCcOCApXv22bP2W3y5evVn3bNDQoxZTpiWUwYDMG+eCjNmaKDVZv2a0KqVAd9/n4KAgPz594P3ifQwJ9LCfEgPc+J8WESTZPAFxEGSk6HYu9dSUG/bZtO605kxBQbC2KkTDKGhMNWvD7stvktW8R5xjLg44Nw5+dMPGc6ft3x+8MBxxbU9VK36fNFsckhX6uhoGcaMccGBA1nPW+rqKuKzz7QYNkxv95cN3ifSw5xIC/MhPcyJ82ERTZLBF5ACYDRCfugQlOHhUG7ZAtnNm7k+lblUKRg7dLCsR928OewyAJTS4T1SsB48ENIV1alFtj3XWs6LoKBn3bObNDGidOmC+fMuisDy5Up89plLtuPQX3nFiB9/TEH16ma7XZ/3ifQwJ9LCfEgPc+J8WESTZPAFpICJImSnTqUV1PKzZ3N/Knd3GNq0gTE0FIZ27YBixewYaNHFe0R6UlK0OHHiLhIS/BEdrUnrFn7+vDzfJzLz8zOjaVMjmjWzFM3ly0vrz/nduwI++sgFmzZlvXSeQiFi7FgdPvxQZ5f33nifSA9zIi3Mh/QwJ86HRTRJBl9ApEV25QqwcSOwcSPcT56EIObupUJUKmFs2tTS7btDB4hlytg50qKD94j0ZJYTUQRu3RLSWq1TP1+4IEdSUu6K6zJlzGnds5s2NSIw0Dn+fG/erMD48S7ZjjWvUsWEWbNS0KhR7iZBTMX7RHqYE2lhPqSHOXE+LKJJMvgCIj2pOQnQaOC2dy+U4eFQ7NsHQa/P9TlNwcEw+/vDXL48RD8/mMuXh/npZ7FMGY6rzgLvEenJaU7MZiAmRkg3kdm5c3JcvCjLMCFXqVLmtCWnmjY1omJFs9PO4xcXB3z+uQuWLs1iKvKnBg/W4fPPtfD0zN21eJ9ID3MiLcyH9DAnzifrmT+IiGAZ72wYOBCGgQOBhAQodu+2dPveuRNCfHyOziU/fx7y8+et7hMVCohlyqQV1RmK7fLlke3itUQSJpMBAQEiAgKMeO21Z9tNJuD6dRnOnZPBYACqVrXM2O2sRfOLvLyAWbNS0LOnHmPHuiA6OvM3yxYvVmPbNiW+/TYFHToYHRckERGRjVhEE1HOeHjAGBYGY1gYUvR6KCIjodiyBcqtWyG7ezdPpxaMRggxMZDFxGR6jLlEifQt2Kmt2KkFt7c3l90ipyOXA0FBZgQF2W+CLSlq2tSEAwcS8b//qTFrlhomk/V79fZtGfr1c0O3bnp8840WPj7sNEdERNLBIpqIck+lgrF1axhbt4Z25kzI//3XUlCHh0N++XK+XFL26BHw6BHkJ09a3S9qNGkFtvhcV/HUglssWxZQZd+llIjyh4sLMHmyDmFhBowe7YqTJzNvld6wQYW9exX46istXn/dwPfHiIhIElhEE5F9yGQw1asHU7160H3+OWQXL1rGUG/ZAsWxYw4LQ9BqIb98OdMiXhQEiKVLP2vBfq6reGqhzdnEifJfrVpm7N6diPnzVZg+XYOUFOsVclycDCNHumLtWiNmzUp2mgnViIio8GIRTUT2JwgwV60KXdWq0H3wAYRbt6DcuhXyI0cgu3EDsps3Idy5A8Hs+K6rgihCuHMHsjt3gCNHrB4jFitmKaj9/dN/+PnBHBBgKbLZJEaUZwoFMHq0Hp06GTF2rAv278/835L9+xVo1MgDEydqMXy4Hgr+B0NERAWEf4KIKN+J5cpBP3QoMHTos40GA4TbtyGLiYHs5s20z8Jz3wspKQUSr/DkCeRPnkB+5ozV/aKn57Px2C8U2qK/P8TixVlkE+VAhQpm/PlnElauVOKzzzSIi7O+HFZKioBJk1ywfr0SP/6Ygpo1C/cYciIikiYW0URUMJRKiAEBMAUEwOqqsKII4eFDCDdvprVeZyi0HzxwdNQAACE+HvL//oP8v/+s7hfd3Z+1XL9QYJv9/SGWKMEim+gFggC88YYBbdsaMWGCBn/+mfncBcePK9CihTvGjNFh/HgdXFwcGCgRERV5LKKJSJoEAaK3N0Rvb5jr1LF+THIyZLduWVqtrRXbt25BMDp+iRwhMRHys2chP3vW6n7R1TVDN3Hx+WKbM4xTEebrK2LJkhRs2WLAhx+64M4d663SJpOA777TYONGJWbNSkGTJlbfjiMiIrI7FtFE5LxcXWGuXBnmypWt7zeZINy7Z7WreNq2HK5zbQ9CcnLW62W7uGRsxfbzg6l0aShkMqB8eQdHTOR4oaFGNGmSgC+/1GDxYnWmx0VHy9GpkzveekuHKVO00GgcGCQRERVJLKKJqPCSyyGWLQtT2bIwNWhg/ZgnT9KPy46JgXDjhqVV+8aNAukyLqSkQH7xIuQXL6bb7gqgBACzqyvEoCCYg4JgevrZXKECzEFBEMuUAWTWW+6InE2xYsC332rRo4cB773ngsuXM18Oa8kSNbZvV2L6dBE1azowSCIiKnJYRBNR0VasGMzFisFco4b1/UlJlgL7ucJaiIl59n1srGPjBSBLTgbOnIH8zBkoX9gnurhYCuqnRfXzhbZYrhwLbHJKjRubcOBAImbOVOOHH9QwGq0Pd7h7V4a33/ZCkyaVUK2aEjKZAqKItA8AMJuRbpsoCi98n/EY4MXHpB6T8bHP77f2OFEE3NyAevWMaNvWiOBgM0dvEBE5GSEuLo4LLpIkaLVaxMTEwM/PDxr2x5ME5sQGycnPWrFv3HjWip1aaN+9W9ARphHVapgDA58V2BUrWorsChUg+vkB8sxb+cg63iOOd+aMDO+954JjxwpHO4Cfnxnt21smVGva1AhX14KOyP54n0gL8yE9zInzKRx/gYiICoqrK8xVqsBcpYr1/VrtsyW7nu8m/rTQFu7cgSA65r1MQaeD/MIFyC9cyLBPVCphDghIa71O9+HnByhfbPMmKhg1apjx119JWLBAha++0iA52bmbcWNiZPj5ZzV+/lkNjUZEs2aWFup27QwICGA7BxGRFLGIJiLKTxoNzJUqwVypkvX9Oh1kt25ZLbBlN25AuH3bIUW2YDBAfvky5JcvZ9gnyuWWCc4qVkzXTdwcFARzQACgynwpIqL8IJcDI0fqERpqwPvvu2Dv3sLxJo9WK2DnTiV27lRi/HgXBAeb0grqhg1NfC+LiEgiWEQTERUktRrmoCAgKMj6etl6PYTbtyG7fh2m6GgknTmDEg8fQnH9OuRXrjhkdnHBZIL86lXIr17NsE+UySwzhz83wZnp1VctE7lxoCfls8BAEevXJ2P1aiUmTtTg8ePCNeb//Hk5zp+XY/ZsNTw9RbRsaSmo27Y1wseHrdRERAWFRTQRkZSpVBADA2EKDIS2QQPcjomBPHXMlChCePQIsuhoyK5csXxcvWr5HB0NWVxcvocnmM0Qrl+H7Pp1YO/etO2mqlWhHzYM+j59LLMoEeUTQQD69TOgTRsjvvlGjfBwBe7elT+3X4QgIN2HTIYM257/SD2vTJbxsdY+UufrS79dtHpNUQSuXpVlOjlaZuLjBWzcqMTGjZbm6Lp1jWjXzvJRt66JcwYSETkQJxYjyeCkCtLDnEhLTvMhPH78rLh+8ePhQwdEDIjFikE/YAB0Q4ZADAx0yDUdifeI9KSkWHLi7y/dnDx5Auzdq8DOnUr89ZcC9+/nrQL29jajTRsj2rc3omVLA7y87BOnvfA+kRbmQ3qYE+fDlmgiokJKLF4cpldegemVVzLujIuD7No1yFOL6ujoZ63Ydly2S3jyBOo5c6CaOxfGDh2gGzYMpmbN2NWb8s3zrclSVawYEBZmRFiYEWYzcOKEHDt3KrBzpyJXs44/eCDD6tUqrF6tglwuokEDE9q3N6BdOy6hRUSUH1hEExEVRV5eMNepA3OdOhn3JSRYCuqrVyF/sav4nTu5upwgilBu3Qrl1q0wVasG3bBhMPTqhUK5ng9RDshkwMsvm/DyyyZ8/LEOsbECdu2yFNR79igRH5+zCthkEnDwoAIHDyrw+eeWJbTatbMU1IV1CS0iIkdjEU1EROl5eMBcqxbMtWrB+OK+pKRnLdZXrz5ryb56FbKbN206vfzsWbiOGQPz55/D8Oablq7efn52fxpEzsjHR8Trrxvw+usGGAwp+PtvOf76S4mdOxU4dy7na7nHxMiweLEaixdbltBq2vTZElqBgRzRR0SUGyyiiYjIdm5uMNeoAXONGhn3JSdDuWULVAsXQnH0aLanksXFQT1rFlSzZ8PYqZOlq3fjxtLvi0vkIEol0KSJCU2amPDFF8D16wJ27bIU1BERCqSk5Oxe0WoF/PWXEn/9pcRHH7mgatX0S2hxtToiIttwLkciIrIPV1cYevVC0q5dSNy1C/revSHasLCtYDZDuWkT3END4d60KZTLlwMpKQ4ImMi5BASIGDxYjzVrknHlSjzWrUvC0KE6+Pubc3W+CxfkmDNHjS5d3FGxoicGDnTFihVK3LvHN7KIiLLC2blJMjgzofQwJ9LijPkQ7t6F6pdfoPr1V8ju37f5ceYSJaB/6y3oBw+GWK5cPkaYN86Yk8KuKOZEFIGLF2VPJydT4tAheY6X0HpR8eJmuLkBHh4i3NxEuLuLcHMD3N3Fpx+Am5tln+UYPD3m2f7U7+VyLW7dKlo5kbKieI9IHXPifNidm4iI8o1YujR0EydC98EHUG7YANWCBVCcOJHt42SPHkHz3XdQz5oFQ5cu0A8bBlODBuzqTWSFIABVq5pRtaoeo0fr8eQJsG/fsyW0YmNz3vHw8WMZHj+2V4TF4OLiDXd3wMMDLxTjLxbnlu+tFePPf69S8eWAiAoOi2giIsp/ajUMffvC0KcP5P/8A9XChVBu3AjBmGHqsnQEkwmqDRug2rABptq1LbN6d+8O8J16okwVKwZ07WpE166WJbROnZJhxw5LQf3vv3KIouOrz5QUOVJSgBx0SMmSQvGsqPbwsLSGP19ku7unbktfoHt6Ztzm7g7Icz5nGxEVYSyiiYjIcQQBpvr1kVK/PrRTpz7r6v3wYbYPlZ88CdcRI2CePPlZV+8yZRwQNJHzksmAOnXMqFNHhwkTdLh/37KE1l9/KbB7txJPnjhnc67RKCAuTkBcnH3O5+qavqhOLcptKc4tRfyz7zUatpITFXYsoomIqECIZctC99ln0H34IZR//AH1ggWQnz6d7eNkDx5AM3Mm1D/8AENYmKWr96uv8r9WIhuUKiWiXz8D+vUzwGhMXULL0vX77Nmi2xybnCwgOVlAbGzezyWXi/D2FtGggQmtWhnQqpUR/v6cgoioMOHEYiQZnFRBepgTaSn0+RBFyA8ftnT13rwZgslk80ONL78M/bBhMISFAWp1/sX4gkKfEyfEnOReTIxlCa3//pMhPl5AUpKAxEQBSUlAYqLw9MPydV4nLitqKlc2oVUrI1q3NiIkxAg3t4KLhfeI9DAnzoct0UREJA2CAFOjRkhp1AjamBhLV+8lSyCzYXYjxbFjUAwbBvOkSdC//Tb0gwZB9PV1QNBEhYefn4hBg/Q2HavXI11RnVpwp//e8nV8vAn37iUD8EBKijytIE8tzlMfW5hduiTHpUtyLFyohkolomFDE1q3trRS16hhZkcaIifDlmiSDL4LJz3MibQUyXykpEC5bh3UCxdC/t9/Nj9MVCph6NYN+nffhenll/MtvCKZE4ljTqTHlpyYzUByMjIpxq1/n5AgZGghT0wUnm5HgUyglhs+Pma0bGlppW7Z0ohSpfL3X3PeI9LDnDgftkQTEZF0ubjAMHAgDAMGQB4VBfXChVBs2QLBbM7yYYLBANXatVCtXQtjvXqWrt5duwJKpYMCJ6KckMmQNlmXr2/ei8jUovz5Iju16H5WfFsvxBMS0h+TmChAp8u/gjw2VoY1a1RYs0YFAKhd29JK3bKlEQ0amKBS5duliSiXWEQTEZH0CQJMTZoguUkTCNevQ714MVRLl0J48iTbhyr++QeKf/551tW7f3+IxYpZ/mt/8YN9KokKheeLciDvRbnBkFpgZyy8X/z+yRMBR44o8N9/uZuo7eRJOU6elOO77yzxN2liaaVu1cqIoCB2/SaSAhbRRETkVMSAAGi//BLaCROedfU+dy7bx8nu3IHmq6+g+eqrrM8vCJkX2E8/i0+3uQsCiosi5AoFBLk83XGitce+8LX44j6ZDGKxYjA2aQLDG29ALF7cXj82IsoDpRIoXlyE5Za0rSi/e1fAnj0K7NmjwN69Cjx8KMvxdRMTBWzfrsT27ZZeNAEB5rSx1M2aGeHpmeNTEpEdcEw0SQbHg0gPcyItzEcmRBHyiAhLV+9t2yCIhePPmujqCn2/ftC/+y7MlSsXdDhOg/eJ9DAnlu7lp07JsHu3Ert3K3DkiDzPM5zL5SLq138263ft2ibIbWj8Zj6khzlxPjl/S4yIiEhKBAGm5s2R/NtvSDh+HLqRIyEWguYZITkZ6sWL4VGvHlx794Zi716gkLxBQFTUyGRAnTpmfPCBDlu3JuHKlXisXJmEIUN0qFDB9uX8nmcyCTh0SIGvvtKgVSt3VK7sgbffdsGKFUrcvs0+30T5iUU0EREVGmJgILRffYX4s2eR8u23MFWpUtAh2YVy5064desG95AQKJctA1JSCjokIsoDT08gNNSImTO1OH48EcePJ2DmzBR06GB4Oo475x49kmH9ehVGjXJFtWqeaNzYHZ99psGePQq+ZBDZGbtzk2SwK4v0MCfSwnzkgihCsW8fVAsWQLljR0FHYzfmkiUtk6QNGcL1sF/A+0R6mJOcMRiAI0fk2LNHgd27FTh5Up7n5bo0GhEhIZbJyUJCkuDqeg3+/syHVPAecT4sokky+AIiPcyJtDAfeSO7cgWK3bshxMZaukWLImA2W5bLevHjuf0Zjnluu8loRHJiItw0GsgFId2+F49N/T6zc0EUIaSk5Hw97B49oBsxAuZatfLxp+c8eJ9ID3OSNw8eCNi3z1JQ792rwN27ee9I6uOjR5MmJrz6KlC3rgm1apng5maHYClXeI84H87OTURERYI5KAj6oCC7njM//vGRnTsH9fz5UK5ZA0Gny/JYwWCAavVqqFavhjEkBLoRI2B87TXYNLsQETkFb28RPXsa0LOnAaII/PefDHv3WorqQ4cUuVrDOjZWhfXrgfXrLd/LZCKqVjWjbl1T2keNGiawniOyjkU0ERGRhJhfegkpP/4I7eTJUP36K1Q//wzZvXvZPk4RFQVFVBRMFSpAP2wY9P37Ax4eDoiYiBxFEIAaNcyoUUOP0aP1SE4GoqIUaUtpXbiQuzfQzGYB587Jce6cHL/9ZtmmUIh46SVLYf3yy0bUqWNCtWpmqFR2fEJETorduUky2JVFepgTaWE+pMchOdHroVy/Hup58yA/dcrmh4mentAPGADdO+9ADAjIn9gkiPeJ9DAnjhMTI2DvXktBvW+fAnFx9p1DWKUSUaOGpaW6Th0TXn7ZhKpVzVCwWS5PeI84H/7KExERSZlKBUPfvjD06QP5wYNQz5sHxdat2a6HLcTHQz13LlTz58PYqRN0I0bA1KCBpSmLiAolPz8RAwcaMHCgASYTcOyYHLt3W4rqo0flMJvzdv/r9QKOHVPg2LFnJYSLi4hatSxFdWpX8MqVzZBxDSAqxFhEExEROQNBgCkkBMkhIRCuXYN6wQKoVqyAkJiY9cPMZig3bYJy0yYY69aFfsQIGMLCAKXSMXETUYGQy4F69UyoV8+Ejz/WIS4O2L9fgb/+EnDoEHD1qkuei2oASEkR8PffCvz997Oywt1dRO3apnRjrCtUMPM9PCo02J2bJINdWaSHOZEW5kN6CjwnT55AtWIF1AsXQnbjhs0PM5cpA/3QodC/9RbEEiXyMUDHK/CcUAbMibSk5qNECX9cvOiK48flOHFCjuPH5bh0Kf8mJSxWTETdusa0ruB165rg5yeysAbvEWfElmgiIiJnVawY9CNHQj9sGBRbt0I9fz4Uhw5l+zDZnTvQfPkl1P/7H/R9+0I/fDjMVao4IGAikgo3NxGNGpnQqJEpbduTJ8DJk8+K6uPH5bh2zT6F9ZMnAvbtU2Lfvme9YEqWTD8jeN26JpQpw/Y9kj4W0URERM5OoYCxSxcYu3SB/PhxqObPh3L9eghGY5YPE1JSoP71V6h//RWGtm2hHz4cxpYtOW6aqIgqVgxo1syEZs2eFdaPHwtpRfWxY5YC++ZN+wx4fvhQhl27ZNi161lhXbq0GXXqWLqht25tQO3a7AZO0sPu3CQZ7MoiPcyJtDAf0iPlnAi3b0O1eDFUv/wC2ePHNj/OFBwM3fDhMPTuDbi45GOEeSSKEOLiINy/b/l48ACy+/dhunMHCY8ewaN4cSjUakChgKhQADIZoFBYBooqFBDlcsvXT7+HXG7Z9twxadue//75c2VyDBQKiC8cA7kcRXWmJSnfJ0WRPfIRGyuka60+flyOe/fy5/e7dGkz2rQxom1bA1q2NMLTM18uU6B4jzgfFtEkGXwBkR7mRFqYD+lxipwkJ0O5di3U8+dDfuGCzQ8zlywJ/aBB0A8ZArF06XwM8DlabVox/GJxLMTGPvv6wQPLvmxa2qVGFARLge3pCVPDhjCEhcHQvj0KZVXwHKe4T4qQ/MiHKAJ37ghpBfWJE5ZW60eP7FtYKxSWLujt2hnQrp0RVaoUjlZq3iPOh0U0SQZfQKSHOZEW5kN6nConZjMUe/ZYunrv3m3zw0SlEobu3aEbPhzmOnVyfM201uLYWMhSi9/ni+OnH7IHDyDEx+fs/IWAqFbD2KqVpaB+7TVLf9pCxqnukyLAUfkQReDGjedbrBU4flyO+Hj7Vb3+/ma0b29A27ZGNG1qlHTnmazwHnE+LKJJMvgCIj3MibQwH9LjrDmRnT8P1YIFUK1eDUGrtflxxsaNoRs+HKbatZ8VxM8Xx08/y562GgsPHkAwmbI/MQEARJUKxpYtLQV1hw6Al1dBh2QXznqfFFYFmQ+zGbh2TYZjx551Az91So7ExLwX1hqNiGbNjGjb1oh27QwICHCeEof3iPNhEU2SwRcQ6WFOpIX5kB5nz4nw8CFUv/4K1c8/Q3b3bkGHQ88RlUpLQd2lC4yhoRCLFy/okHLN2e+TwkZq+TCZgMuXZenGV584IYden7fCumpVU1pB3bChCSqVnQLOB1LLCWWPRTRJBl9ApIc5kRbmQ3oKTU70eig3bIB63jzIT54s6GjoBaJCAWOLFpaCulMnp1vbu9DcJ4WEM+QjKQmIiFBg504F/vpLmefZwD08RLRsaZmcrG1bI0qXllb54ww5ofS4xBUREVFRp1LB0KcPDL17Q37oENTz5kGxZQsEUTr/aIqenjB7e0MsVQpiqVIwlyoF8cXvS5VCirs7bj58iPJlykCjUFiauYzGtM+C2Wz53tq21GPNZsukZanfW9tmMlm6qj9/fpMpwzFpx6V+r9NBERUF+aVLNj93wWiEctcuKHftgvj++zA2awZDWJilhdrbOx9/6kQFw80N6NDBiA4djBBFLc6dk2HnTgV27lTi77/lMJly1kqdkCBg0yYlNm2yLKVVq9azycleecUEuX2WwqYihEU0ERERWQgCTI0bI7lxYwjXrkG9cCFUK1ZASEiw+6VEhcJSAHt7w+zjk1YQP18cp2339gZsbJ0RtVqYk5MhenhAtPIYSbwtIIqQnTsH5Z9/QrlxY45mTRdMJij37oVy716I48bB1KSJZQx1p04QS5XKx6CJCoYgANWqmVGtmh5jx+oRFwfs3avEzp0K7NqlwP37OW+lPnXKMhZ75kygRInUJbSMaN3aiBIlJPEqQRLH7twkGezKIj3MibQwH9JTJHISHw/VihVQL1wI2fXrWR4qFiuW1iKcoSBO/drHB+ZSpSyzUOfD2jTOmBPZ+fOWgnrTJsjPns3VOUSZDKaQkGcFta+vnaPMvTzlxGSCEBMDeXQ0ZJcuQXb5MmR37sBcrhwMHTvC1Lx5vvweFWbOeI9kxmwGTpyQP22lVuDYsby1D8pkIurVM6FdO0vX75o1HbOEVmHKSVHBIpokgy8g0sOcSAvzIT1FKicmExT79kF25gwgkz0rjp8WxqK3N6Qwc4+z50R28SKUGzdC+eefkP/3X67OIT7tUWDo2hWGLl0ct853JmzKSVwc5JcupRXK8suXLQVzdDQEnS7Tc5uqVYNuxAgYevUC1Op8egaFi7PfI1mJjRWwe7eloN69W5nn5bTKlDGjbVtLQd2ihREeHnYK9AWFOSeFFYtokgy+gEgPcyItzIf0MCfSU5hyIrt8+VlBffp0rs4hCgJMDRs+K6jLlrVzlNlLy4mvL1zv3LEUytHRlqL58mXL9w8f5ukaZh8f6IcMgf7ttzlOPBuF6R7JitEIHDkiTxtLffZs3gY+K5UiGjc2oW1bA9q3N6JSJfu1UheVnBQmLKJJMvgCIj3MibQwH9LDnEhPYc2J7MqVZwV1HmZQNzZo8KygLl/ejhE+JYoQ7t591pp86RJw4QLEixehvn0739cNFzUaGPr0gW74cJiDg/P1Ws6qsN4j2bl5U8Bff1nGUu/fr0Byct4q4MBAE1q1MqJ5cyOaNTOhePHcl1RFNSfOjEU0SQZfQKSHOZEW5kN6mBPpKQo5Ea5dsxTUGzdCcexYrs9jrFfvWUHt75+zByclpSuUZU/HLMujo/NlIrrcMLRpA/3IkTC2aMFx088pCvdIdrRa4OBBRdpY6itX8tZKLQgiatc2oUULI5o3N6FhQyNcXHISD3PibFhEk2TwBUR6mBNpYT6khzmRnqKWE+H6dSg3bbIU1EeP5vo8xldeeVZQBwZaNqZO6pVaKF++/KwL9u3b9nkCuSC6uEBISbH5eFO1atANH24ZN10EfieyU9TuEVtER8vSCuqoKAX0+ry96aJWi2jQwITmzY1o0cKIOnWyXkaLOXE+LKJJMvgCIj3MibQwH9LDnEhPUc6JEBPzrKA+ciTX5zHVqAGYTJBduZLlpF75SXR3h7liRZgqV4a5UiWYK1eGqWJFmCtVApRKKH//Heq5c3M0m7m5VCnoBw+GfvDgIr0cWFG+R2yRmAhERFgK6r/+UuLWrZwvofWiYsVENG1qTCuqXxxPzZw4HxbRJBl8AZEe5kRamA/pYU6khzmxEG7ehHLzZktBffhwQYdjlSiTwRwQAHNqoVypEkxPC2axdOnsu2CLIuQREVDPnQvlzp22X1ethqF3b+hGjID5pZfy+Cycj+TvEYMB8lOnIJYoAXNgYIF2xRdF4L//ZGljqY8ckcNkyns85cqZ0ayZ8Wn3byO8vFKknRPKgEU0SYbkX9SLIOZEWpgP6WFOpIc5yUi4fTutoJYfOgRBdOy/fuYSJZDk5wfFSy8BwcFpLcvmwEC7LUslu3gRqvnzoVq9OkddvQ2tWlnGTbdqVWTGTUv2HjGZoFq2DOrp0yG7f9+yqWJFGLp0gbFLF5jq1CnwHMXFAXv2KPHXX5bJyW7fznsrNQBUrWpEnToPERqqQosWMnh62uW0lI9YRJNkSPZFvQhjTqSF+ZAe5kR6mJOsCXfvQhkebpnl++BBCGazXc4rqtUwBwVZWpOfa1k2V66MFBcXh+VEePgQql9/heqnnyC7d8/mx5mCg5+tN52TGaGckBTvEXlEBFw++STLtdHNfn4wdOkCQ9euML36KiCzTwGbW6IIXLokw/79Cuzbp0BkpCLP61IDgFwu4tVXTWkt1fXqmaBS2SFgsisW0SQZUnxRL+qYE2lhPqSHOZEe5sR2Qmzss4L6wAGbCmpzuXLPCuWKFS1jlStVgujnh8xmTiqQnOh0UK5fD/W8eTlaY9tcsqRl3PSQIRB9fPIxwIIjpXtEuHYNLpMmQbl5c44eZy5bFoZOnWDo0gWmRo0y/d1zJKMROHlSjn37LEX133/L8zxBGQC4uopo3PjZeOrq1c0F/f4BgUU0SYiUXtTJgjmRFuZDepgT6WFOcke4fx+KLVugDA+H7No1iB4e6VqTTZUqwVyxIuDmluNzF2hORBHyyEjLuOkdO2x/mEoFQ69elnHT1avnY4COJ4l7JCEB6u++g3ruXAh6fZ5OZS5VCoZOnWDs0gXGJk0ApdJOQeZNcjLw99+KtKL61CkZRDHvRbW3d/rx1AEBLOUKAotokgxJvKhTOsyJtDAf0sOcSA9zIj1SyYns0iWoFiyA6rffcjZuukULy7jp1q0LvAuxPRRoPsxmKFetgubLL3PU3d7m0xcvDmPHjjB07Qpj8+Z2G3NvD48eCYiMfNZSffWqfVrPAwMt61O3aGFE06YmlCzJ0s4RWESTZEjljyw9w5xIC/MhPcyJ9DAn0iO1nAiPH0O1ZAlUixZBdueOzY8zValiGTfdp49Tj5suqHzIDx+G5pNPoDh+PNtjTVWqwBwcDMWuXRCSk3N1PdHTE4bXXrNMTNa6teRydv26gP37FWljqh8+zPsbNIIgomZNc1ordcOGxtx0HiEbsIgmyZDaH1liTqSG+ZAe5kR6mBPpkWxO9HooN2ywjJs+edLmh5lLlID+7behHzoUoq9vPgaYPxydDyEmBpopU6D6449sjxWLFYP2k0+gHzzY0i07ORmK3bst659v3w4hISFXMYhubjC0bQtj164wtG0LuLvn6jz5JTlZi337HuDiRX9ERWkQFaVAcnLeu34rFCJeftmEkBAjGjc2oUEDI2f+thMW0SQZkv0jW4QxJ9LCfEgPcyI9zIn0SD4nogh5VBTU8+ZBsW2bzUuAiUolDD17WsZN16yZz0Haj8PykZQE9axZUM+enW33eVEmg/7tt6H75BOIJUtaP0ing2LfPsva51u3QhYXl6uwRI0GxtatLTN9v/YaUKxYrs5jTy/mRK8Hjh61dP3ev1+Bo0ftsz61TCaidm0TQkIshXWjRkZ4eeU9/qKIRTRJhuT/yBZBzIm0MB/Sw5xID3MiPc6UE1l0tGXc9MqVOepGbGzWDLqRI2Fs21by46bzPR+iCOXvv0MzZQpkt25le7ixeXOkzJgBc7Vqtl/DYIAiMhKKTZssk+E9eJC7UJVKGFu2hKFzZxhDQyGWKJGr8+RVdjmJjwcOHrR0+46IUODsWfuMpxYEETVqmNG4sREhIUaEhHBMta1YRJNkONMf2aKCOZEW5kN6mBPpYU6kxylzEhcH1dKlUC9cCNnt2zY/zFS5MvTDh0Pfty/g6pqPAeZefuZDfuwYNB9/DMWRI9kea6pQAdpp02Ds2BEQ8tDKajJBfvCgpcv35s2Q3b2bq9OIcjmMTZtaunyHhjp0ibOc5uTuXQEREYq0lupbt+z3xs1LL5nSCurGjY3w9WWpaA2LaJIMp/wjW8gxJ9LCfEgPcyI9zIn0OHVODAYoN26Eau5cmybESmUuXhyGXr1g6NQJpsaNAYUiH4PMmfzIh3DnDjRffgnVqlXZHit6eEA7fjz0w4bZf/Zssxnyf/6xFNQbN0J282auTiMKAkyNGsHQtSsMnTtDLFvWvnG+IC85EUUgOlqWNut3ZKQCT57kvet3qkqVnnX/Dgkxolw5lo4Ai2iSEKf+I1tIMSfSwnxID3MiPcyJ9BSKnIgi5IcPQz13LhRbttg8bhqwTERm7NDBspZxy5ZAAf8M7JoPrRbquXOh/u47CElJWR4qCgIMb7wB7aRJjmnlFUXIT5yAYuNGKDdtgvzKlVyfyli/PgydO8PQpQvEgAA7Bmlht5yYTDCl6HHmmAlHDoqI+scVB4644kGCGoB9CuvAQFNaK3VIiGWd6rx0JHBWLKJJMgrFH9lChjmRFuZDepgT6WFOpKew5UR29apl3PSKFdkWji8S3dxgbNMGhk6dYGjXrkAmtbJLPkQRik2b4DJpEmQ3bmR7uLFRI8u45zp1cne9vBJFyP77D8qNG6HcvBny8+dzfSpjnTowdukCQ+vWlh4GBgMEvR4wGJ59rddDMBiAF79+bv/zX5u0WiQ/fgw3lQpykyntfBmOfXoeQa8HjMaM1zGbM49broIeKqSY1dCKauihgg5q6JD+6xe/z+5rVy8lAiorEFRNgSo1lPApr4CgUUNUqQC1GqJSaelx8NzXokpleTNJQut45xSLaJKMwvZHtjBgTqSF+ZAe5kR6mBPpKbQ5iYuDavlyy7jpXHQbFpVKGJs3t7RQd+zosDG4ec2H7NQpuHzyCRRRUdkeay5fHtqpU2EIC8vbuGc7k124YOnyvWkT5KdPF3Q4RZKxUSMkbdtW0GHkmrSnDyQiIiIikiIvL+hHj0bC8eNI/uUXGF95JUcPFwwGKHftguvYsfCoWhVur70G1ezZEK5dy59480i4fx+asWPh3rx5tgW06OoK7aefIuGff2Do1k1SBTQAmKtWhW78eCRGRiLh+HGkfPFFjvNHeaRSFXQEecIimoiIiIgot5RKGLp3R9KuXUjcsQP6fv1gzuHiu4IoQnH4MFwmTYJnnTpwb9IE6q+/huzMGcvMUQVJr4dq9mx4vPIK1EuWZDseXN+7NxKOHoVu/HjAxcVBQeaeuUIF6MeMQdLu3Yg/fRop06fD2LAhRIkV/oXNtTsa3LvnvD9j6UwVSERERETkrAQBpgYNkNKgAWAwWJZdCg+HcsuWHC2TBQDyM2cgP3MGmq+/hikwEMZOnSwzfdev77h1qEURih07oPn0U8ijo7M93Pjqq9DOmAFTvXoOCC5/iH5+0I8YAf2IERDu3rXkb+NGyKOishxvTDl36qIrfGIFp11Ci0U0EREREZE9KZUwNW8OU/Pm0H7zDeTHj0OxeTOU4eGQX76co1PJr12DfM4cqOfMgdnHB4aOHWHs3BnGpk3zrUus7Px5aCZOhHLPnmyPNZcpA+2UKTD06uW4At8BxNKloR8yBPohQyA8eADF1q1QbtwIxf79EIxG+11HLgdUKpjkcsieTsAFpdIy+VbqZ2tfK5WAUgmoVGnHpvtaLn82EZlOZ5mcTKcDdLpnk5M9tw86neXr5/ZBp7NMXJb62Z6USlSv7rxvTLCIJiIiIiLKLzIZTK+8AtMrr0D3+eeWSa3Cw6EID4fixImcnSo2FuolS6BesgSipycM7dtbJiZr3Rpwd89zqMLjx1DPmAHV4sUQTKYsjxU1GuhGjYJu7Fi7XFvKRG9vGAYOhGHgQCAuDorISMgePLAUss8Xrs8Vv1Cp0vZn+vXTYtcpJt8zm5/NAv60wE4tzB/dsSyr9d8xIy6cMuLuDSNU0KfN9/3816nfq6pXRWsnfs+FRTQRERERkSMIAszBwdAFB0P34YcQbtyAcssWSwv1oUM56jIsxMdDtW4dVOvWQdRoYGzRAobOnWHs0AFiiRI5i8tohOqXX6CeMQOyx4+zPVwfFgbtF1/ky5rJkuflBWPnzgUdhePJZJZlqTQaPN8BWwTgVRVo0gJo8nTbo0cCDh6UIypKgagoBU6flkEU049/ntYzBYDeMbHnAxbRREREREQFQPT3h374cOiHD7d0Gd62zdJKvW+fpWutjQStFsrt26Hcvh2iXA5T48aWtahDQyGWL5/lYxV79kAzcaJNayebatVCyowZMIWE2BwbFT0lSojo1MmITp0s3d7j4oDDhy0F9cGDcpw4IUdISNY9HaSORTQRERERUQETvb1hGDAAhgEDgIQEKHbvhnLzZih37oSQkGDzeQSTCYrISCgiI+EyYQKMdeumTUxmrlo17Th5dDRcv/wSyh07sj2nuVQpaCdNgqF/f8tYW6Ic8PICXnvNiNdesxTVCQmAq2vBxpRXLKKJiIiIiKTEwwPGsDAYw8KQotNBERFhaaHeuhWy+/dzdCrF8eNQHD8OzdSpMFWpAsVrr6H8gwcouW5dtpNFiUol9MOHQ/vhh4CnZ16eEVEaD4+CjiDvWEQTEREREUmVWg1j27Ywtm0LfPcd5EeOWJZe2rwZshs3cnQq+cWLcLt4EW42HGvo2BHaadNgDgrKXdxEhRiLaCIiIiIiZyCXw9SoEUyNGkE7bRpkp09bCurwcMjPnrXLJUwvvWQZ99yihV3OR1QYsYgmIiIiInI2ggBzrVrQ1aoF3cSJkF25AkXqTN9HjkAQxezP8Rxz8eLQffop9G+9BShYIhBlxYlX5yIiIiIiIgAwBwVBP3o0knbsQMK5c0j57jsYWrWCmE1BLMrl0A0bhsRjx6AfMoQFNJENeJcQERERERUiYunS0L/9NvRvvw3ExUG5c6dlYrJduyAkJ6cdZ2jdGtrp09PN2k1E2WMRTURERERUWHl5wdC7Nwy9ewMpKTDv3o34kyfh3qoVFA0bAoJQ0BESOR0W0URERERERYGLC/Rt2uB+1arQ+PlBwQKaKFc4JpqIiIiIiIjIRiyiiYiIiIiIiGzEIpqIiIiIiIjIRiyiiYiIiIiIiGzEIpqIiIiIiIjIRiyiiYiIiIiIiGzEIpqIiIiIiIjIRiyiSVLkcnlBh0AvYE6khfmQHuZEepgT6WFOpIX5kB7mxLkIcXFxYkEHQUREREREROQM2BJNREREREREZCMW0UREREREREQ2YhFNREREREREZCMW0UREREREREQ2YhFNREREREREZCMW0UREREREREQ2YhFN+erYsWPo1asX/P39UbZsWbRp0wYbNmzI0Tl0Oh2++eYbvPzyy/D19UVwcDDGjBmD+/fv51PUhc/t27cxb948dOvWDTVq1ECpUqVQpUoVDBgwAEePHrX5PJGRkfDy8sr0Y+XKlfn4LAqfmjVrZvqzDA0NzdG51q5di1atWqFs2bIICAhAnz59cOLEifwJvJBauXJllr/fXl5e6NKlS7bn4X2Sc2vWrMHYsWPRokUL+Pj4ZPtzio+Px8SJE1GjRg34+PigZs2amDRpEhITE3N87d27d6Njx44oX748/Pz80KlTJ+zfvz8vT6dQsDUnBoMBGzduxLvvvov69eujXLlyKF++PFq3bo3FixfDZDLl6Lr2fF0sbHJyn8yYMSPL16Hr16/n6NqXL1/GW2+9haCgIJQuXRohISFYvHgxRLFor5Sbk5xk9/fFy8sLN2/etOm6vE8KnqKgA6DCKyIiAj169IBGo0H37t3h7u6OTZs2YdCgQbh58yZGjx6d7TnMZjNef/117N69G/Xq1UOXLl0QHR2NZcuWYf/+/di1axe8vb0d8Gyc26JFi/DDDz+gQoUKaNmyJby9vREdHY0tW7Zgy5Yt+Pnnn9G9e3ebzxcSEoImTZpk2F6zZk17hl0keHp6Yvjw4Rm2+/v723yOmTNnYtq0afDz88OgQYOQmJiI9evXo3379ti4cSMaNmxoz5ALrZo1a2LChAlW923atAnnzp1D69atbT4f7xPbTZs2DTExMShZsiR8fX0RExOT6bFJSUkIDQ3F6dOn0apVK/Ts2ROnTp3C7NmzERUVha1bt0Kj0dh03TVr1mDYsGHw9vZGv379AAAbNmxAWFgYlixZgq5du9rl+TkjW3Ny9epVvPnmm3B3d0ezZs3QoUMHxMfHY/v27fjggw+wc+dOrF69GoIg2Hxte7wuFkY5uU9S9evXz+rPrVixYjZf9/z582jXrh20Wi3CwsJQpkwZ7Ny5Ex988AHOnz+P//3vfzl6HoVJTnKS2d+Xq1evYu3atQgODkb58uVtvjbvk4IlxMXFFe23kChfGI1G1KtXD7dv38Zff/2FWrVqAQCePHmC1q1b48aNGzh69Gi2N/qKFSswatQo9OzZEz/99FPaH+FffvkF48aNw1tvvYUffvghv5+O09u0aRNKlCiR4R/6gwcPomvXrnBzc8OFCxegVquzPE9kZCQ6d+6MCRMm4JNPPsnPkIuE1GLq9OnTuT5HdHQ0GjRogMDAQOzevTvtH6NTp06hbdu2CAwMxKFDhyCTseNRbun1egQHByM+Ph5nz56Fj49PlsfzPsm5ffv2ISgoCP7+/vj+++/xxRdfYO7cuejfv3+GY6dPn47/+7//w9ixYzFlypS07VOmTMEPP/yAyZMnY9y4cdleMy4uDrVr14ZCoUBERATKlSsHALh16xaaNWsGADhx4gQ8PDzs8ySdjK05uX37NrZu3Yp+/frBzc0tbXtSUhI6deqE48ePY8mSJQgLC7PpuvZ4XSyscnKfzJgxA9988w02b96Mpk2b5um6HTt2xMGDB7Fu3Tq0bdsWgOV1sWvXrjh06BB27tyJ+vXr5+kazionOcnM+PHj8dNPP2HatGkYNWqUTY/hfVLw+F8V5YuIiAhcvXoVPXv2TCugAcs7n+PGjYNer8eqVauyPc+yZcsAAJMnT073LvagQYMQGBiIdevWISUlxf5PoJDp0qWL1Raxxo0bo2nTpoiLi8PZs2cLIDLKq5UrV8JoNOKDDz5I17JQq1Yt9OjRAxcuXMChQ4cKMELnt2XLFjx69Ajt27fPtoCm3GnRooVNrSeiKGL58uVwd3fH+PHj0+0bP3483N3d0/5uZOfPP//EkydP8M4776QV0ABQrlw5DB06FA8fPkR4eHjOnkghYmtOypYtiyFDhqQroAHAzc0NI0eOBABERUXlS4xFja05safLly/j4MGDaNq0aVoBDQAqlQqffvopAGDp0qUOjUlK8poTrVaLdevWQaVSoW/fvnaMjPIbu3NTvjhw4AAAoFWrVhn2pXaHzO6PqlarxdGjR1G5cuUML1CCIKBly5b49ddfcfz4cTRu3NhOkRc9SqUSACCXy21+zJUrVzBv3jxotVqULVsWzZo1Q9myZfMrxEJNr9dj5cqVuHv3Ljw8PPDyyy/j1Vdftfnx2d1rv/32G6KiohASEmK3mIua1KJs4MCBOXoc7xP7i46Oxp07d9C6dWurRVuDBg2we/du3Lx5M9tukdndO19//TWioqLSunlTzuXm7wuQ99dFeubgwYP4999/IZPJEBQUhBYtWsDd3d3mx2d1nzRq1Ahubm58kyQPNm/ejLi4OHTt2jXHwxN5nxQsFtGUL6KjowEAFStWzLDP19cX7u7uuHLlSpbnuHr1KsxmM4KCgqzuT90eHR3NIjqXYmJisG/fPpQuXRrVq1e3+XHr1q3DunXr0r5XKBR45513MHXq1Bz/s1TU3bt3L621JtXLL7+MxYsXo0KFCtk+Pjo6Gu7u7vD19c2wL/X+S70fKedu3LiB/fv3o1y5cmjTpk2OHsv7xP5Sf5ez+ruwe/duREdHZ1tEZ/V3iveOfaxYsQKA9QIsK3l9XaRnZsyYke77YsWK4euvv7b5zaGs7jm5XI6AgACcP38eRqMRCgXLipxavnw5gJy/SQvwPilo7M5N+SI+Ph6AZdIDazw8PNKOye4cmU1+kXru7M5D1hkMBgwbNgw6nQ5Tpkyx6Z96b29vTJkyBYcOHcKtW7dw6dIlrFy5EkFBQZg3bx4mT57sgMgLj/79+2Pjxo24dOkSbt++jYiICPTp0wfHjh1Dly5dkJCQkO054uPjs7zPUo+h3Fm5ciXMZjP69etnc+HL+yT/2PPvQlZ/p3jv5N2SJUvw119/oVmzZmjXrp3Nj7PH6yIBNWrUwJw5c3DixAncvXsXJ0+exP/93/9BEASMGDECW7dutek82d1zHh4eMJvNuZoZv6i7du0aIiMjUb58ebRs2TJHj+V9UvD4lhFREWQ2mzFixAgcPHgQb775ps3jcF566SW89NJLad+7ubkhNDQUr776KkJCQrBw4UKM/f/27j0oyuqNA/gXBJVAAVGgYLnIxSAVhSwuo0ajOUEUFzWJEBshZJAo08jGSv3RqHmbiRTFGR0wLuYFcdNUXFEYQVJnFGyVhVFRKZBQkIXi/vvD2a2VZdnFBRW+nxlm5HnPed/zsp7dfd7znvd8+inGjRvXX00fVL788kuF3ydPnoydO3cCePTU4NTUVLUfMkLa19nZifT0dOjo6ODDDz9Uux77CQ11x48fx4oVKyAQCJCSkqJRXb4vakdAQIDC77a2tvj4448xYcIEBAYGIjExEX5+fk+pdQQ8ulOjq6sLYWFhGj/8k/3k6eNINPWL3kYDGhsbexw9e3wfDQ0NSrf3NtpNynV2diI2Nhb79+/H/PnzsXXr1ifep4WFBfz8/NDe3q7RutOk3EcffQQAKC4u7rXs6NGjVfYzWRnS3JkzZ3D37l3MmDEDdnZ2T7w/9pMnp83PBVWfU+w7fXfy5ElERETA3NwcQqEQlpaWWtmvJu+L1LOZM2fC3t4eYrFYrTsteutzjY2N0NHR0WieNT36LpaZmQldXV2NLtL2hv1k4DCJpn6haj5ZTU0NpFJpj3PaZOzs7KCrq9vj3GlZXNl8NlJONgKdmZmJuXPnIjk5WWtLH5mZmQEAmpubtbK/oUyTv6WDgwOkUilqamq6bVM155N619cHiqnCfvJkZP+XtfG5oOpzin2nb06cOIHw8HCYmZlBKBRq5eKTDPuO9sj+luqsbqKqz3V0dKCyshK2tracD62hU6dOoaqqCr6+vhAIBFrbL/vJwGESTf1C9iTg06dPd9smEokUyvTEwMAAHh4eKC8vx+3btxW2dXV1IS8vD4aGhpg6daqWWj24yRLorKwsBAcHY+fOnVp9uJFsZG2gl98YjDT5W2qjr1F39+/fx7Fjx2Bqaop33nlHa/tlP3kyDg4OePHFF1FcXIympiaFbU1NTSguLoatrW2vDxUD2He07cSJE1i4cCFMTU0hFAp7vVCuKfYd7WhqasL169dhaGgoT7hUUdVPioqK0NTUxH7SB0/yQDFV2E8GDpNo6hczZ86EnZ0dDhw4gJKSEnm8oaEBW7Zs6bYeXnV1NSQSSbfbhSIiIgAAa9euRVdXlzy+Z88e3Lp1C/PmzYOBgUE/n83zT3YLd1ZWFgIDA5GSkqIyga6rq4NEIkFdXZ1C/PLly0rLJycno6CgAA4ODnB3d9dm0wctiUSi9EqxRCLB6tWrAQBz586VxxsaGiCRSFBdXa1QPiwsDHp6eti8ebNC/ykpKcHBgwcxYcIEeHl59c9JDGJZWVlobW3F/PnzMWLECKVl2E8Gno6ODsLDwyGVSrFx40aFbRs3boRUKpV/bsg0NzdDIpHgzp07CvGgoCCMHj0aKSkpqKqqkserqqqwa9cumJmZafUCymCWm5uLhQsXwsTEBEKhsNcR/La2NkgkEty8eVMhrun7IinX2NiIioqKbvG///4b8fHxaGxsRGBgYLfRY4lEAolEohBzcnKCt7c3CgoKkJubK4+3trbiu+++A6D9RHCw++uvv3D8+HGMHTsWb7/9do/l2E+ebTr19fVdvRcj0lx+fj5CQkIwcuRIBAcHw8jICEeOHMGdO3fwv//9D3FxcfKyMTExyMzMxLZt2xAWFiaPd3Z2Yt68eRCJRJg2bRp8fHxw48YNCIVC2NjYQCQSabyu3lC0bt06bNiwAUZGRliyZInSBNrf3x+TJ09WKJ+QkICVK1fKy0yaNAn6+vqYOnUqXnrpJTQ3N+PChQsoKSmBsbExDh06BA8PjwE7r+fZunXrsH37dnh7e0MgEOCFF15ARUUFcnNz0dbWhmXLlik8xTk9PR2xsbEIDQ1FcnKywr42bdqExMRECAQCvPvuu5BKpTh06BBaW1uRk5MDT0/PgT695563tzfEYjHOnTvX4/Jv7Cfak5aWhqKiIgCAWCzGlStX4OnpKV+mxcvLS/5FvampCXPmzMHVq1fx5ptvws3NDVeuXMHp06fh7u6Oo0ePKlxcLSgoQEBAAHx8fHD06FGF4+7btw/R0dEYO3YsgoKCAADZ2dmoq6vDnj17EBgYOABn/2xS9zWRSCSYPn06WlpaEBISAkdHx277srGxUfhsr6yshJubGwQCAUpLS+VxTd8Xhxp1X5PKykpMmTIF7u7ucHZ2hoWFBe7du4ezZ8+iqqoKrq6u+OWXXzBmzBiF/ZuYmAAA6uvrFeLXrl3DnDlz8M8//yAoKAiWlpY4efIkrl27hqioqG4XtIYSTd67ZJKSkvD1118jNjZWfiFCGfaTZxsnMFC/mTFjBo4fP45169YhOzsbbW1tcHV1xZo1axAcHKzWPnR1dZGRkYGtW7di37592L59O0xNTREeHo5Vq1YxgVaT7HZ4qVSKTZs2KS1jY2MjT6J7snjxYohEIhQWFuL+/fvQ1dWFQCBATEwMli5dCisrK623fbCaPn06JBIJSkpKUFRUhObmZpiZmWH27NmIjIzUaF3V5cuXw8bGBsnJydi9ezf09fXh5eWFr776ClOmTOm/kxikLl26BLFYDA8PD43WT5dhP9FcUVERMjMzFWLnz5/H+fPn5b/LvogaGhri6NGjWL9+PYRCIQoKCmBhYYGlS5ciISFBo7uT3n//fZiZmWHz5s3IyMiAjo4O3NzcsGLFCrzxxhtaObfnlbqvSU1NDVpaWgAABw8eVLovHx8fhSS6J9p8XxyM1H1NTE1NERkZiUuXLiE3Nxf19fUwMDCAs7MzoqOjERUVpVE/cXFxgUgkQmJiIk6ePInm5mY4ODhg06ZNWLx4sdbO73mkyXuXjGz99L6O4LOfPBs4Ek1ERERERESkJs6JJiIiIiIiIlITk2giIiIiIiIiNTGJJiIiIiIiIlITk2giIiIiIiIiNTGJJiIiIiIiIlITk2giIiIiIiIiNTGJJiIiIiIiIlITk2giIiIiIiIiNTGJJiIiIiIiIlITk2giIiJ6blVWVsLExAQmJiZPuylERDRE6D3tBhAREVH/8vf3x7lz59QqW19f37+NISIies4xiSYiIhoirK2tYW1t/bSbQURE9FxjEk1ERDREhIWFYeXKlU+7GURERM81zokmIiIiIiIiUhOTaCIiIurm8Qd2/frrr/D394etrS2srKwwa9Ys/Pzzzyr3UVZWhtjYWEyaNAnm5uawtbWFn58f0tLS0NHR0WO99vZ2ZGRkIDg4GI6OjjA3N4eLiwv8/PyQlJSEhoaGHusWFRVh/vz5sLe3h6WlJby9vZGSkoKuri6l5S9fvoyoqChMnDgR5ubmsLKywqRJkxASEoKkpKQe6xER0dDF27mJiIhIpZ07dyIhIQGmpqYYP348qqqqcPHiRfnP999/361OdnY2oqOj0draCkNDQ7i6uuLBgwcoLCxEYWEhsrOzkZGRAQMDA4V6dXV1+OCDD1BcXAwAsLCwwMSJE1FbW4vi4mIUFhZiypQpmD59erdjpqenIy4uDsbGxrCzs8OdO3cgFovxxRdf4Pbt20hMTFQof+rUKYSGhqKtrQ1GRkZwdHSEnp4e/vjjD4hEIohEIsTExEBPj1+XiIjoXxyJJiIiIpVWrVqFFStWoLy8HHl5eSgrK8OWLVugq6uLlJQUHD58WKG8RCJBTEwMWltbsXDhQkgkEpw5cwZXrlzB4cOHMXr0aOTl5eGbb75RqNfV1YWIiAgUFxfD2toaOTk5KCsrw+nTp1FaWoobN25g8+bNMDc3V9rOZcuWITExERUVFcjLy0NFRYX8GNu2bcPNmzcVyq9evRptbW2Ij49HeXk5CgsLkZ+fj4qKCpSWlmLNmjXQ1eVXJSIiUqRTX1/P+5SIiIgGMXWXuPLz80NGRgaAR7dzu7m5AQB8fX2RnZ3drXx8fDxSU1Ph6uqKwsJCeXzp0qX46aef4OrqinPnzkFHR0ehXlpaGj755BPo6+ujtLQUlpaWAB7dMh4aGooRI0YgPz8fEyZM6LXN/21neHg4kpKSupXx9vaGWCzG+vXrsWTJEnncwsICLS0tqKyshLGxca/HIiIiAjgSTURENGRYW1vD09Ozx5+XX35Zab2YmBiVcbFYjLt378rjubm5AIAlS5Z0S6ABIDQ0FOPGjUNbWxvy8vLk8SNHjgAAAgIC1EqgHxcZGak0/tprrwEAbty4oRAXCAQAgAMHDmh8LCIiGro4yYeIiGiI6OsSVy4uLkrjTk5O0NPTQ3t7OyQSCaytrdHQ0ICamhoAgKurq9J6+vr6cHJyQm1tLcrLy+VxsVgM4N+kV1OOjo5K4+PGjQMASKVShXh8fDzi4uLw+eef48cff4Svry+mTZsGHx8f2NjY9KkNREQ0+HEkmoiIiFTqaQ7ysGHDMGbMGABAY2MjAMVEtad6AOS3cMvq/ffffb212tDQUGlcNq/58Sdth4eHY+/evXj99ddRWVmJ3bt3IyYmBpMnT8asWbNQUFDQp3YQEdHgxiSaiIiIVLp3757SeEdHB+7fvw8AGDVqFADAyMio13oAUF1drVDvv/9WtYSVtgUEBODEiRO4efMm9u/fj88++wz29va4ePEiQkJCUFpaOmBtISKi5wOTaCIiIlLp+vXrSuPl5eVob28HADg7OwN4NIpsYWEB4N/bsx/X3t4uv41bVg8AXnnlFQDAb7/9pp2Ga8DY2BizZ8/Gt99+iwsXLmDatGlobW1FWlragLeFiIiebUyiiYiISKUdO3aojLu6usLa2loef+utt+TbH7+FGgCysrJQW1sLfX19+Pr6yuPvvfceAEAoFCrMlR5oenp68PDwAAD8+eefT60dRET0bGISTURERCrl5+djw4YN8lHnrq4upKamYu/evQCA5cuXK5SPi4vDyJEjIRaL8emnn6KpqUm+7ezZs1i1ahUAYNGiRfJRawCYM2cOZsyYgZaWFgQHB3ebk/zw4UPs3r0bZWVlT3xODx8+REREBEQiEVpbWxW2Xb58Wb6kl7u7+xMfi4iIBhc+nZuIiGiISE9Px9mzZ1WW2bBhg3zdZZnExEQkJCRgx44dsLe3R1VVlfwJ3JGRkQgODlYo7+zsjOTkZERHRyM1NRUHDx6Ek5MTHjx4gFu3bgF4tPb02rVrux1/z549WLBgAS5cuICAgABYWlrCysoKtbW1qKqqQkdHB4RCYZ+WwPqvzs5O5OTkICcnB8OHD8f48eNhaGiI2tpa3L59GwDw6quvKqwrTUREBDCJJiIiGjLu3r2rsJ6zMg8fPuwWi46OhkAgwLZt21BaWor29nZ4eHggKioKCxYsULqfoKAguLi44IcffkB+fj5+//13jBw5El5eXggNDUVYWBiGDRvWrZ6ZmRmOHTuGzMxM7N+/H1evXkVpaSnGjh0LT09P+Pv7d0vy+2LUqFHYtWsX8vPzcenSJVRXV6OhoQGjRo2Cl5cXgoKCsGjRIgwfPvyJj0VERIOLTn19fffJSkRERDSkVVZWypPV+vr6p9sYIiKiZwjnRBMRERERERGpiUk0ERERERERkZqYRBMRERERERGpiUk0ERERERERkZr4YDEiIiIiIiIiNXEkmoiIiIiIiEhNTKKJiIiIiIiI1MQkmoiIiIiIiEhNTKKJiIiIiIiI1MQkmoiIiIiIiEhNTKKJiIiIiIiI1MQkmoiIiIiIiEhNTKKJiIiIiIiI1MQkmoiIiIiIiEhN/wdAZezJew2P9wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1000x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = sbs_emb.plot_losses()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ZCKFSECRPIpW",
    "outputId": "fcd1e427-f1cf-42b3-c556-78fdec492f5f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[419, 453],\n",
       "        [285, 318]])"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "StepByStep.loader_apply(test_loader, sbs_emb.correct)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_kqa22QNPIpX"
   },
   "source": [
    "## Model II - GloVe + Transformer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "id": "MH3DdVEFPIpX"
   },
   "outputs": [],
   "source": [
    "class TransfClassifier(nn.Module):\n",
    "    def __init__(self, embedding_layer, encoder, n_outputs):\n",
    "        super().__init__()\n",
    "        self.d_model = encoder.d_model\n",
    "        self.n_outputs = n_outputs\n",
    "        self.encoder = encoder\n",
    "        self.mlp = nn.Linear(self.d_model, n_outputs)\n",
    "\n",
    "        self.embed = embedding_layer\n",
    "        self.cls_token = nn.Parameter(torch.zeros(1, 1, self.d_model))\n",
    "\n",
    "    def preprocess(self, X):\n",
    "        # N, L -> N, L, D\n",
    "        src = self.embed(X)\n",
    "        # Special classifier token\n",
    "        # 1, 1, D -> N, 1, D\n",
    "        cls_tokens = self.cls_token.expand(X.size(0), -1, -1)\n",
    "        # Concatenates CLS tokens -> N, 1 + L, D\n",
    "        src = torch.cat((cls_tokens, src), dim=1)\n",
    "        return src\n",
    "\n",
    "    def encode(self, source, source_mask=None):\n",
    "        # Encoder generates \"hidden states\"\n",
    "        states = self.encoder(source, source_mask)\n",
    "        # Gets state from first token only: [CLS]\n",
    "        cls_state = states[:, 0]  # N, 1, D\n",
    "        return cls_state\n",
    "\n",
    "    @staticmethod\n",
    "    def source_mask(X):\n",
    "        cls_mask = torch.ones(X.size(0), 1).type_as(X)\n",
    "        pad_mask = torch.cat((cls_mask, X > 0), dim=1).bool()\n",
    "        return pad_mask.unsqueeze(1)\n",
    "\n",
    "    def forward(self, X):\n",
    "        src = self.preprocess(X)\n",
    "        # Featurizer\n",
    "        cls_state = self.encode(src, self.source_mask(X))\n",
    "        # Classifier\n",
    "        out = self.mlp(cls_state) # N, 1, outputs\n",
    "        return out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "id": "aUum393hPIpX"
   },
   "outputs": [],
   "source": [
    "torch.manual_seed(33)\n",
    "# Loads the pretrained GloVe embeddings into an embedding layer\n",
    "torch_embeddings = nn.Embedding.from_pretrained(extended_embeddings)\n",
    "# Creates a Transformer Encoder\n",
    "layer = EncoderLayer(n_heads=2, d_model=torch_embeddings.embedding_dim, ff_units=128)\n",
    "encoder = EncoderTransf(layer, n_layers=1)\n",
    "# Uses both layers above to build our model\n",
    "model = TransfClassifier(torch_embeddings, encoder, n_outputs=1)\n",
    "loss_fn = nn.BCEWithLogitsLoss()\n",
    "optimizer = optim.Adam(model.parameters(), lr=1e-4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "id": "2mBjsLwHPIpX"
   },
   "outputs": [],
   "source": [
    "sbs_transf = StepByStep(model, loss_fn, optimizer)\n",
    "sbs_transf.set_loaders(train_loader, test_loader)\n",
    "sbs_transf.train(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 395
    },
    "id": "4CwpEol1PIpY",
    "outputId": "0655742f-0ada-4c42-f4fd-945f099c7110"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9EAAAF6CAYAAAAedNt3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACTv0lEQVR4nOzdd1xV9f8H8Nc5d3DZiCgOQNyGu7LcmlrmIBVnWZZpKqlpVtqwtG+m1a/coxxlpplmWQ4cmSnOTE1NyZGIgLhQARmXO875/YGMy71XQC53wOv5eNyH8Pmc8b744XLf97OElJQUGURERERERERUJNHRARARERERERG5CibRRERERERERMXEJJqIiIiIiIiomJhEExERERERERUTk2giIiIiIiKiYmISTURERERERFRMTKKJiIiIiIiIiolJNBEREREREVExMYkmIiIiIiIiKiYm0URERERERETFxCSaiIiIiIiIqJiYRJPT0Gq1iI2NhVardXQoRFaxnZIrYDslV8B2Sq6CbZUKYxJNTsVoNDo6BKIisZ2SK2A7JVfAdkqugm2VCmISTURERERERFRMTKKJiIiIiIiIiolJNBEREREREVExMYkmIiIiIiIiKiYm0URERERERETFxCSaiIiIiIiIqJiUjg6AiIiIiIjySZKEjIwM7kvsJCRJglqtRmpqKu7evevocKiENBoNPD09IYq26z9mEk1O5Z136iAgwB3NmwNNmhjRtKkRfn6OjoqIiIjIPiRJwq1bt+Dl5YWAgAAIguDokCo8SZKg0+mgVqttmohR2ZNlGVqtFrdu3ULlypVt9v/HJJqchlYL/PFHJRiNAn74Ib88KEhC48Y5CXWTJkY0biyhTh0JCoXjYiUiIiIqCxkZGfDy8oK7u7ujQyFyeYIg5P0uZWRkwNvb2ybXZRJNTuP8eSWMRvNPWxMTRSQmitixQ5VX5uEh46GHjGjSREKTJrnJtRE+PvaMmIiIiMi2tFotAgICHB0GUbmi0WiQnJzMJJrKnzNnit8cMzMFHDumxLFjpuW1apkm1U2bSqhVSwJH3hAREZGr4BBuItuy9e8Uk2hyGv/+W/rmePmyiMuXRWzdmt9r7e0tIyzMeC+5zkmyw8KM8PQs9e2IiIiIiKiCYRJNTsPXV0ZIiBYJCW6QZdt9WnT3roA//1Tizz/zm7sgyKhTR0Ljxvk9102aGBEcLIMf/hIRERERkTVMoslpvPFGBgYNSoC/fwhiYz1w+rQCp0+LOH1agTNnFEhPt112K8sCLl5U4OJFBTZtyu+19vWV0bhxflLdtKmERo2M4NoeREREROWTn58f2rVrh61btz7wNfbt24fw8HBMmTIF77zzjg2jI2fEJJqciurmTfhkZ6NVqyZo1cqYVy5JQHy8gH/+UdxLrnMely/bdrJzaqqAgweVOHgw/1dDFGXUqyeZDAdv0sSI6tXZa01ERERkC34l3NM0JSWlTOIoT5o2bYobN27g+vXrjg6l3GESTU4lcPVqBHz/PfTduyM7MhLGTp0AQYAoAqGhMkJDDQgPN+Qdn5YGnDmTn1SfOSMiJkaBzEzbZbeSJOD8eQXOn1fg55/zy/39JbPVwRs1kuDmZrNbExEREVUIU6ZMMStbsmQJ0tLSLNbZ0pEjR0q9pdgjjzyCI0eOoHLlyjaKipyZkJKSIjs6CCIAyL55E5VbtIAiIyOvzPjQQ8iOjIR+4EAUd0y10QhcuiTmDQX/55+c4eCJiWW/RLdSKaNBg4LzrHO+rlqVv2blhVarRUJCAoKDg6HRaBwdDpFFbKfkCthOLbt58yaqVKni6DCcQtOmTZGQkODwXmdJkqDT6aBWqyG60JYv7Ik2ZcvfLddpBVTuaX74wSSBBgDFv//C47XX4N2kCdxmzIBw7VqR11EogHr1JPTta8DUqdlYty4Tp0/fxaVLadiyJR2ffJKF55/XoUULA9zcbJvcGgwCYmIUWL9ejQ8+cEdEhCcaNPBBgwbeiIjwwAcfaLBunQpnzojQ6216ayIiIqJy7/Lly/Dz80NkZCTOnTuHoUOHonbt2vDz88Ply5cBAJs3b8aIESPQsmVLVK9eHSEhIejRowd+/fVXi9f08/NDr169TMoiIyPh5+eHuLg4fPXVV2jfvj2qVauGJk2a4JNPPoEkSSbH79u3D35+fpg1a5ZJedOmTdG0aVOkp6djypQpaNSoEapWrYq2bdtajefy5csYPnw4QkNDUbNmTfTs2RMHDhzArFmz4Ofnh3379j3oj8+qjIwMzJw5E61atUJgYCBCQ0MxaNAgHD582OxYrVaLBQsWoF27dggJCUGNGjXQtGlTvPTSS/jnn3/yjpMkCatWrUKXLl0QGhqKatWqISwsDIMHDy6T52BPHM5NzsFohMfy5VarxVu3oPn8c7jNmwd9v37IfvVVSC1alOgWlSrJaN/eiPbt8+daGwzAf/+JZouYXb1q28+XbtwQsXu3iN2788vUahkNG5quDt60qQR/f/ZaExEREd3PpUuX8OSTTyIsLAzPPfccbt++DbVaDQD43//+B5VKhdatW6NatWpITk7Gtm3b8OKLL+LTTz/F6NGji32fDz74AAcOHEC3bt3QpUsXREVF4ZNPPoFer8f7779frGsYDAZEREQgJSUF4eHhyMrKws8//4yXXnoJP/30E7p06ZJ3bFJSErp3745r166hW7duaNasGS5cuIB+/fqhY8eOJfshFZNWq8UzzzyDY8eOoXnz5oiMjMSNGzewceNG/P7771ixYgX69u2bd3xkZCQ2btyIxo0b47nnnoObmxuuXLmCffv24e+//0bTpk0BAB9++CHmzZuH2rVrY+DAgfDy8kJSUhIOHz6MvXv3okOHDmXyfOyBSTQ5BfHMGSiK0css6PVQr18P9fr1MLRpg+zISBh69crpfn4ASiXQqJGERo0kDBiQX56cLODMGdFkIbNz50To9baba63T5SyU9s8/prHXqCHlzbHOHQ5er570oE+RiIiIypEnn/R0dAhW/fZbRtEH2cjhw4cxefJkvPvuu2Z1P/74I0JDQ03K0tPT8dRTT+Hjjz/GCy+8AA8Pj2Ld5+TJk9i3bx/8/f2hVqsxZcoUPPzww1i6dCmmTJmSl7jfz9WrV9GyZUts2bIl7/iBAweiT58+WLRokUkSPX36dFy7dg3vv/8+3njjjbzy7777DuPHjy9WzCU1b948HDt2DIMGDcJXX30F4d7KuaNHj8aTTz6JCRMmoGvXrvD29kZqaip++eUXtGjRAr///jsUBd6gGo1G3L17N+/7VatWoXr16jhw4IDZz/vOnTtl8lzshUk0OQWpWTPcPHYM+gULUG3jRihu3CjyHOWhQ1AeOgQpJATZo0ZB98ILgK+vTeIJCJDRqZMRnTrl91rrdMD582KBhcxyeq5v3rRtr3VSkoikJBE7d+ZvvaXRyHjoIaPZQmYlXMiSiIiIXNxff/HtOwAEBgbizTfftFhXOIEGAC8vLzz33HOYOnUqjh8/jvbt2xfrPm+99RaqVasGnU4HAKhcuTJ69uyJtWvX4sKFC2jcuHGxrjNz5kyThLtTp04IDg7G8ePH88qys7Px66+/okqVKhg3bpzJ+c8//zzmz5+PCxcuFOt+JbF27VqoVCpMmzYtL4EGgObNm+PZZ5/Ft99+i61bt2LIkCEQBAGyLEOj0ZjND1coFGarrKtUKpNEO1elSpVs/jzsib+F5DTkgABcHTkSynffhfe2bXBbvBiKU6eKPE+Mj4f71KnQfPIJdM89B92YMZDq1LF5fGo17iWxEgYPzp/QfP26YJJUnz6twPnzIoxG2/Vaa7UC/v5bib//Ni0PCpIKDAXPSbJr15bgQmteEBEREZVYkyZNrPYC37x5E3PmzMGuXbuQkJCArKwsk/prxRj9mKuFhemDNWvWBACkpqYW6xq+vr4WE/uaNWviyJEjed9fuHAB2dnZaNmyJdwKbfciCAIee+wxmyfRaWlpiIuLQ8OGDfOeV0EdOnTAt99+i3/++QdDhgyBj48PnnrqKezcuRMdO3ZE37590b59ezz88MNQqVQm5/bv3x/Lly9HmzZtEBERgQ4dOuCxxx4r9UrozoBJdDmwadMmfP311zhx4gRSUlJw8uRJ1KpVy9FhPTg3N+iHDIF+8GAoDh2C2+LFUG7dCkG+/1xhIT0dbkuXQr1sGQxPP52zRVaHDijrzZwDA2UEBhrQtWt+mVYLnDsnmuxpffq0iDt3bJvdJiaKSEwUsX17/ouWp6eMsLD81cHbtjXgoYek+1yFiIiIyLVYW2X5zp07eOKJJ5CYmIjWrVujU6dO8PX1hUKhwD///IOoqChkZ2cX+z7e3t5mZbk9q0aj0azOEh8fH4vlCoXCZIGy3KHQAQEBFo+vWrVqse5XErn3tPbzDAwMNDkOAFauXInZs2fjxx9/xEcffQQg5zk+99xz+OCDD/KGbn/yySeoVasW1qxZg88//xyff/45NBoN+vbti48//tiltwNjEl0OZGZmom3btujZsycmT57s6HBsRxBgbNsWmW3bQoiLy0mQv/sOQoFfYounyTJU27ZBtW0bjI0b52yRNWAAYMftMzQaoHlzCc2bSwByeq1lGUhKEkz2tD59WoH//hMhSbZL9DMyBPz1l9JkuNczz+ixYEGmrUa7ExERETmUYKWT5LvvvkNiYiLee+89vPXWWyZ1c+bMQVRUlD3CeyC5CXtycrLF+hvFmO74oPe8efPmfe9Z8MMEDw8PTJ06FVOnTkVcXBz27duHb775Bl9++SW0Wi3mzp0LAFAqlRg/fjzGjx+Pq1ev4sCBA1izZg1++OEH3LhxAz///LPNn4+9MIkuB4YMGQIAiImJcXAkZUcODYV25kxo334b6jVroP7qKyji4oo8T3HmDDzGjYP04YfQvfwydCNGQC6DT/GKQxCAmjVl1KxpQPfuhrzyzEzg7NmcnurchczOnFEgLc12ifWmTSrExnphw4YMVKvG1b+JiIhcVatWhqIPqsAuXboEAOjZs6dZ3aFDh+wdTonUr18fbm5uOHHiBLKzs02GdMuyjL/++svm9/Tx8UFoaChiY2ORlJSEGjVqmNTv378fAPJW3C4sNDQUoaGhGDBgAOrXr49t27blJdEFVa9eHQMGDEBERAQeffRR7NmzB1lZWS47tNvlkujNmzdjxYoVOHnyJDIzMxEYGIhWrVrhww8/RFBQkN3iWLduHQ4dOoQTJ04gJiYGOp0OixYtwtChQ62ec/z4ccyaNQt//vknDAYDwsLCMHbsWPTr189ucbs8Hx/oIiOhGzUKyu3b4bZkCZT3frnvR7x5E5pPP4XbnDnQ9++P7MhISM2a2SHgonl4AA8/bMTDDxtRsNc6Pl4osIhZTpIdG/vgS3SfPq3AU0954eefM1CvHod3ExERuSJ7roDtioKDgwHkrN5dcNGvH3/8ETt37nRUWMXi5uaGPn36YP369ViyZAkmTpyYV7d27VqcP3++TO777LPPYtasWfjwww/x5Zdf5vXynz59Gt9//z18fHzy9tFOTk7GjRs3EBYWZnKNlJQUZGdnw9/fH0DOImknTpzA448/bnJcRkYGMjIyoFKpzBYmcyUuk0TLsozXX38dK1euRO3atdG/f394eXnlDQ1ISEiwaxI9Y8YMJCQkoHLlyggMDERCQsJ9j4+Ojkb//v2h0WgQEREBLy8vbNq0CcOHD0diYmKZLVlfbikUMPTqBUOvXhBPnYLbkiVQ/fQThHsrJ1oj6HRQr10L9dq1MLRvn7NF1tNPP/AWWWVFEIBatWTUqmVAz575nzinpwMxMaarg8fEKJCeXrxe6/h4Ed27e2L9+kw88kjx5vEQERERuYrBgwdj7ty5mDx5Mvbt24fg4GCcPn0ae/fuRXh4ODZv3uzoEO/rgw8+wJ49ezB9+nQcOHAgb5/oHTt2oFu3bti1a1eJkk+9Xo/IyEir9UuWLMGECROwc+dOrFu3DufPn0enTp1w8+ZNbNy4EQaDAV999VXecO6kpCR07NgRTZo0QePGjVGjRg3cvn0bUVFR0Ov1eTlNVlYWunfvjnr16qFFixYICgpCRkYGtm/fjuvXr2P8+PFmi6e5EpdJor/88kusXLkSI0eOxKeffmq2VLrBUPTQlh9++AFt27ZFSEiIxXqj0YglS5Zg1KhRRe75tmDBAtSpUwchISGYM2cOPvzwQ6vHGgwGTJgwAaIoYuvWrWh2rwd08uTJ6Nq1Kz766CP06dPHJK7p06dbHApRUEpKyn3rKwqpWTNkLVkC7fTpUH/9NdQrVkC0MpekIOX+/VDu3w9jaCh0o0dD9/zzgIXFI5yJlxfw2GNGPPZYfgIsScDlyyL++cd0IbP4eMsvsLduiQgP98SqVZno1o1DwoiIiKj8qFmzJrZu3Ypp06Zhz549MBqNaNasGTZu3IjExESnT6KDgoKwc+dOTJ8+Hbt378aBAwfQvHlz/Pzzz/jll18AWF7szBpJkrB27Vqr9UuWLIFGo8GmTZswd+5cbNy4EYsXL4a7uzvatWuHSZMmoU2bNnnHh4SE4O2330Z0dDT27t2L27dvo3LlymjevDnGjBmDbt26AQA8PT3x4YcfYu/evTh06BBu3rwJPz8/1KtXD9OmTUP//v0f7AfkJISUlBSnnyCZlZWFhx56CH5+fjh69CiUypLn/leuXMHDDz+MwMBAbNmyxSyRliQJY8aMwfr16/HZZ59h1KhRxb52bhJtbTj37t27ERERgaFDh2LRokUmdd9//z1effVVvPPOO5gyZUpeeXJyMm7fvn3f+zZo0MDk+5iYGLRt29ZlV+fWarVISEhAcHAwNKVZBEyrheqnn3K2yDpzptinyT4+0D3/PLJHjYJsYRsCV3P5soBBgzxx7pzlXnalUsbChVkYMkRvsZ4ss1k7JSpDbKfkCthOLbt586bVlZLJMSRJgk6ng1qtdugQ5KeffhpHjhxBfHw8vLy8HBaHq7Ll75ZLDETfvXs3UlJS0KtXLxiNRmzatAlz5szB119/jdjY2GJdo2bNmli+fDmSkpIQHh5uMvy6YAI9ZMgQjBw50qbx507I79Kli1ld13v7Ih04cMCkPCAgAA0aNLjvg6zQaKAfOhTp+/cjfdMm6Hv0gFyMba6EtDS4LV4M74cfhsfzz0Nx4EDO5GQXVauWjG3bMvDYY5Z7mw0GAWPGeGD+fLUrP00iIiKicsXSPtbr1q3D4cOH0blzZybQTsAlhnOfOHECQM5eau3atcN///2XVyeKIl599VXMmDGjyOuEh4djxYoVGDFiBHr37o0tW7agZs2aiIyMxPr16zFo0CAsXrzY5p8wXbx4EQBQt25ds7rAwEB4eXkV+8MAS+7cuYOEhIS81QjPnTuH1NRUBAcHo1KlSg98XZcnCDB27IjMjh0hxsZC/eWXUK9ZAyHj/gtyCJIE1ZYtUG3ZAmPz5jlbZEVEAEUM8XdG/v4yfvklA8OHe2DHDpXFYz74wB3XromYMUMLF17fgYiIiKhcaNOmDZo1a4aGDRvm7W+9f/9+eHt75+3LTI7lEm+Zc/dKW7RoEXx8fLB7924kJiYiKioK9erVw8KFC7FixYpiXatPnz5YunQpEhMTER4ejhEjRmDdunXo378/lixZUiZDNNLS0gBY32jd29s775gHERUVhY4dO+LFF18EAAwaNAgdO3Z06n3w7E2qUwfazz5D2pkzyJoxA9K9lRuLojh5Eh5jxsC7aVO4ffYZhGLMtXY2Hh7AmjWZeP5564uuLV7shtGj3VHEumxEREREVMZefvll3Lx5Ez/88AOWLVuG//77DwMHDsTvv/9usuI4OY5LJNGSlLMdj1qtxpo1a/Dwww/Dy8sLbdu2xcqVKyGKIhYuXFjs60VERGDJkiWIi4vDxo0b0atXLyxdutRssTJXMXToUKSkpJg97rfdVoXl5wfduHG4+/ffyFi1CoYCCyXcj3j9OjQzZ8K7cWO4jx8PsQRzrZ2BUgksWJCFN9/UWj3mxx/VGDzYA3fv2jEwIiIiIjLx/vvv4+DBg4iPj0dycjLOnj2LZcuWcTqnE3GJJDq3B7dFixaoXr26SV1YWBhCQ0Nx6dKlYq9WLcsyoqOj874/e/Ysbty4YbN4C8uN31pv8927d632UlMZUSpheOYZZGzbhvQ//oBu0CDIxViwTsjOhvq77+Ddrh08+/SBcvv2nOWxXYAgAFOnZuOzz7IgCJYnQf/xhwrh4Z64ebN4W2YREREREVU0LpFE169fHwDg6+trsT63XKu13suWS5ZlTJgwAatXr0ZERASWLl2KuLg4hIeH4+rVq7YLuoDcudC5c6MLun79OtLT01GnTp0yuTcVzdiyJbKWLsXdf/6B9s03Id3bJL4oyr174TlkCLxatYJ62bKcTZxdwKhROnzzTSbUasuJ9IkTSjz1lCcuXXKJlwciIiIiIrtyiXfJHTp0AACcP3/erE6v1yM2Nhaenp4ICAi473VkWcbEiROxatUq9OvXD8uWLcOgQYPw1Vdf4dKlSwgPD7e4Gl5ptWvXDkDOKuOF/f777ybHkOPI1asje+pU3D1zBpnz58P40EPFOk9x8SLc33oLPmFh0Lz/PoQCK787q759DdiwIQPe3pYT6UuXFHjqKU+cOOESLxFERERERHbjEu+Qa9eujS5duiA2NharVq0yqZszZw5SU1PRq1ev++4fLcsyJk2ahG+//TYvgc6dAz1gwACTRPr69es2jb9Tp04IDQ3Fhg0bcOrUqbzy1NRUzJ49G2q1GkOGDLHpPakU3N2hHzYM6QcPIv2XX6B/6qlinSakpcFtwQJ4t2gB95deguLPP516i6yOHY3YujUdgYGWh6PfvCmid28v7N3rmmsFEBERERGVBSElJcV53+UXcOnSJTz11FO4efMmunfvjvr16+PUqVOIjo5GcHAwdu3ahcDAQKvnJyUloVOnTmjXrh2WL19uMeH+8ccfMWbMGCxevBiDBw++bzyrVq3CoUOHAAAxMTE4efIkWrdujdq1awPIWZp+2LBhecdHR0ejf//+0Gg0iIiIgJeXFzZt2oSEhAR89NFHGD9+/IP8WMoVrVaLhIQEBAcHQ6PRODocE+KFC1B/9RXU338PITOz2OcZHn4YushI6Pv2BVSWt5hytLg4Af37e+LiRcvJskol48svs9C/v97OkTknZ26nRLnYTskVsJ1advPmTVSpUsXRYVABkiRBp9NBrVaXyU4+ZB+2/N1ymSQaABITEzFz5kz8/vvvuH37NgIDA9GjRw9Mnjy5WD+QuLg4BAUF3bfH+r///kO9evWKvFZkZCTWrl1rtf7ZZ5/FkiVLTMqOHTuGWbNm4ciRI9Dr9QgLC8PYsWMRERFR5P0qApf4Y5qSAvW338Jt6VKIV64U+zSpenXoXnkFupdeglzMOdf2lJwsYNAgDxw/bv13Y9asLERGcg8sl2inVOGxnZIrYDu1jEm082ESXT5U2CSayjeX+mOq10O1ZQvUS5ZAeeRIsU+TNRrohgyBbswYSI0alWGAJZeeDrz4ogd+/916j/nrr2vxwQfZECrw4t0u1U6pwmI7JVfAdmoZk2jnwyS6fLDl7xZbAdGDUKmg79cPGTt3In3XLuj694dcjH3GBa0WbitXwrt1a3hEREC5a5fTbJHl5QX88EMmBg2y3ts8Z44Gr77qDj1HdhMRERFRBcUkmqiUjI8+iqwVK3D31CloX38dkp9fsc5T7d4NzwED4NW6NdRffw1kZJRtoMWJSQV8+WUWxo/PtnrM2rVqDB3q4QzhEhERERHZHZNoIhuRa9ZE9rRpuHvmDLJmz4axQYNinac4fx7ukybBu3FjuE2fDqEEc63LgigCH32kxYwZWVaP2blThT59PHHrVgUe101EREQuZc2aNfDz88OaNWtMyps2bYqmTZuW+jq2NGvWLPj5+WHfvn1ldg96cEyiiWzN0xO6l19G+uHDyNiwAfquXYt1mpiSAs3cufBu1gzuI0ZAcfRoGQd6f+PG6bB0aSaUSsvLJhw9qsTTT3siPp6JNBEREZXOyJEj4efnhw0bNtz3uLS0NFSvXh0hISHIyrL+gb+z27dvH/z8/DBr1ixHh1IsuUn9Tz/95OhQnAKTaKKyIoowdOuGzJ9+wt3Dh5E9fDhkd/ciTxOMRqh/+gle3brB88knofr5Z8BgsEPA5gYN0mP9+kx4elpOpC9cUKB7dy+cOcOXEiIiInpwL7zwAgBg9erV9z3up59+QlZWFgYMGAD3YryvKo5NmzZh06ZNNrmWrYwaNQpHjhzBI4884uhQyAK+8yWyA6lRI2jnzMHdM2eg/eADSNWrF+s85V9/wePll+HdogXU8+YBKSllG6gFXboYsGVLBgICLC+AdvWqiB49vHDgQNELqxERERFZ0rFjR9SqVQvR0dFISEiwelxukp2bdNtC7dq1Ubt2bZtdzxYqV66MBg0awMPDw9GhkAVMoonsSPb3R/akSbh76hQyly+H4eGHi3WemJgI92nT4BMWBs0bb0C8cKGMIzXVsqURO3ZkoFYty4l0WpqAiAhPbNpkfZ9pIiIiImsEQcDQoUMhSZLVucb//vsvjh07hsaNG6Nly5ZITU3F3Llz0bNnTzRq1AhVqlRBo0aNMHr0aFy6dKnY97Y2J/rOnTt4/fXX0bBhQ9SuXRtdu3bF5s2brV7nu+++w7PPPoumTZsiMDAQoaGhiIiIQHR0tMlxs2bNQnh4OADg008/hZ+fX97j8uXLecdYmxO9bds29O7dGyEhIahWrRratWuHhQsXwlBo5OLly5fh5+eHyMhIxMbGYujQoahVqxZq1KiBPn364J9//in2z6ikihsjAERHR2PAgAFo1KgRqlativr166NHjx5YuXKlyXEnTpzAsGHD0KRJE1StWhV169bFE088gc8//7zMnoc1fMdL5AgqFfQDBkDfvz8Uf/0F9eLFUG3aBKGI7a6EzEy4rVgBtxUroH/qKegiI2Ho3Bn22Li5bl0JO3emY8AAT/zzj3mvc3a2gBdf9MAXX2jx8svWt8kiIiKiB+f55JOODsGqjN9+K9X5zz33HD755BN8//33mDJlCoRC729yk+vcXujz589j5syZ6NChA3r37g0PDw+cP38eGzZswM6dO7F3716EhIQ8UCyZmZno1asXYmJi0KpVK7Ru3RrXrl3Dyy+/jC5dulg856233kKTJk3QuXNnBAQEICkpCVFRUejbty++++479OrVCwDQvn17xMfHY+3atWjXrh3at2+fdw1fX9/7xrVw4UJMnToVlSpVwoABA+Dh4YFt27Zh6tSpOHToEFavXm32c4uPj0e3bt3QqFEjPP/887h06RKioqIQHh6OI0eOoGrVqg/0M7JFjDt27MCQIUPg6+uLnj17olq1akhOTsbp06exbt06vPTSSwCAU6dOoXv37lAoFOjZsyeCg4ORmpqKs2fPYuXKlXjzzTdt+hyKwiSayJEEAcbHHkPWY49BGx8Pt+XLoV65EkJaWpGnqnbuhGrnThgfegjZY8ZAP2gQYKO5QdYEBsrYujUdzz/vieho85cPWRYwaZI7rl0T8M472fbI7YmIiCoU5V9/OTqEMhMUFIQuXbpg165diI6ORqdOnfLqDAYD1q9fDzc3NwwePBgA0KBBA5w7dw6VKlUyuU50dDT69u2Lzz//HPPnz3+gWObNm4eYmBi8+OKLmDNnDnQ6HdRqNYYMGYL+/ftbPOfw4cMIDQ01Kbt27RqeeOIJfPDBB3lJdIcOHQAAa9euRfv27fHOO+8UK6ZLly5h+vTpqFKlCv744w8EBQUBAN5//3307dsXW7duxbp16zBkyBCT8w4cOIDp06dj4sSJeWUzZszA559/jjVr1uD1118v1v3LIsbVq1dDlmVs3rzZbDTA7du3875et24dsrOzsWbNmryfo6Xj7IXDuYmchBwSAu3//oe0mBhk/d//wVi3brHOU/z7LzwmTMjZImvGDAhXr5ZpnD4+wI8/ZqBfP+u9zZ99psHrr2sctR4aERERuShrC4xt374dN27cQM+ePfOSZl9fX7MEGsiZX92oUSPs2bPngeP44YcfoFar8e6775qUd+3a1SS5L6hwAg0A1apVQ3h4OC5evIj4+PgHjgcAfvzxRxgMBowbNy4vOQUANzc3TJ8+HQDw/fffm51Xq1YtvPbaayZluT/n48ePlyomW8VoaZE4f3//Bz6urDGJJnI2Xl7QvfIK0v/6Cxnr1kHfuXOxThNv34bm88/h3bQp3EeNgnjiRJmF6OYGrFiRhdGjs60es3KlG4YN84AL7z5BREREdtazZ08EBARgy5YtSE1NzSu3tqDYvn378Nxzz6Fhw4YICAjIm1scExODa9euPVAMaWlpuHz5MurUqYPAwECz+jZt2lg8Ly4uDq+99hpatGiBwMDAvFiWLl0KAA8cT65Tp04BgMnw71yPPfYYNBqNxXnOTZs2hSiapn01a9YEAJOfsS2UNMbcXv1u3brhrbfewubNm3Hr1i2zc/v16wdRFPH8889j7Nix2LBhA5KSkmwae0kwiSZyVqIIQ/fuyPzlF9w9cAC6F16A7OZW5GmCwQD1+vXw7twZnj16QPnrr0ARc60fMDx88okW06ZprR4TFaVCRISnIxYVJyIiIhekUqkwePBgZGVl5e0Zff36dezatQtBQUHoXKBz4ZdffsEzzzyDffv2oXXr1oiMjMTkyZMxZcoUBAcHQ6d7sDVa7t69CwAICAiwWG9pDnFsbCw6d+6MNWvWIDQ0FMOHD8dbb72FKVOmoF27dgCA7GzrnQ8liatKlSpmdYIgoEqVKnnHFOTt7W1WplTmTMszGo2liqm0Mfbt2xdr1qxBWFgYvv76a7zwwguoV68ennnmmbyEHAAeffRRbNmyBW3btsWGDRswcuRIhIWFoUuXLmYLt9kD50QTuQCpcWNkLVgA7bRpUH/zDdTLl0O8fr3I85SHDkF56BB0/foha8kSQKOxaVyCALz+ejaqVpXw2mvuMBrNJ0EfOqREjx5e2LAhAzVrWt5vmoiIiIrH0KqVo0Mocy+88AIWLVqE7777DiNGjMC6detgMBgwdOhQkx7VTz75BBqNBnv27EHdQtPgfv755we+f27SmZycbLH+xo0bZmWLFy9GSkoKvvrqq7w527lef/11HDhw4IHjKRzXzZs3zRZMk2UZN2/etJgw29ODxNirVy/06tULd+/exZ9//onNmzfju+++w4ABA3DkyBH4+fkBANq2bYu2bdsiKysLR48exfbt27FixQoMHjwYhw4dsjicvqwwiSZyIXJAALLfegvZEyZA9fPPcFu8GIoCn9JZo964EXL16tDOnFkmcQ0dqkdAgIyXXvJAVpZ5Iv3vvwp07+6Fn37KQMOGtu8VJyIiqihKuwK2K2jUqBFatWqFv/76C6dPn8aaNWvytsAq6NKlS2jUqJFZAn3t2jXExcU98P19fHxQq1YtxMbG4vr162a9qocOHTI7J3dLrZ49e5qUy7KMP//80+x4hSJnp5OS9AQ3a9YMW7Zswf79+/HII4+Y1B09ehRarRaPPfZYsa9XFkoTo7e3N7p164Zu3brBaDRi9erVOHbsGLp27WpynLu7Ozp06IAOHTrA19cXM2fOxB9//IHhw4eX2fMqjMO5iVyRWg39kCFI37sX6VFR0IeHQxbv/+us/vJLKI4dK7OQunc3YNOmDFSqZDlJTkwU0b27J/7803x7LCIiIqKCcuc+v/nmmzh37hw6d+5s1rMZHByMS5cumfQMa7VaTJo0CXq9vlT3Hzx4MHQ6HWYW6oDYvXs39u7da3Z8cHAwgJwVuguaM2cOYmJizI7PXRDtypUrxY5p4MCBUCqVWLRoEa4WWEhWp9Nh2rRpAHK2CXOkksZ44MABix8k3Lx5E0DOgmQAcOTIEWi15lMICx9nL+yJJnJlggBj27bIbNsWQlwc3JYuhfq77yBYmA8jSBLcx49H+p49gFpdJuG0amXEjh0ZiIjwRGKieVKfkiKib19PfP11Jnr04NLdREREZFm/fv3wzjvv5CWlhRcUA4BRo0Zh8uTJ6NixI5555hkYjUb88ccfkGUZTZo0wenTpx/4/hMmTMCWLVvw7bff4t9//8Xjjz+Oa9eu4ZdffkH37t2xY8cOk+OHDx+ONWvWYNiwYejbty/8/f1x9OhRnDx50uLxDRo0QPXq1fHzzz/Dzc0NNWrUgCAIGDVqlNW9omvXro3p06dj6tSpaNeuHfr16wcPDw9s374dFy5cQM+ePc2GktvaihUrsGvXLot1w4YNQ5s2bUoU45QpU3Dt2jW0bt0aISEhEAQBhw8fxrFjx9CqVau8Rdzmzp2L/fv3o02bNqhVqxY0Gg1OnjyJvXv3IjQ0FL179y7T510Yk2iickIODYV25kxo334b6jVroJk2DUKhBTUUMTFwmz8f2WW4IX2DBhJ27kzHgAGeiIkx73XOyhLw/PMemDs3Cy+8ULpPiYmIiKh88vb2zlt0qlKlSmZ7AwPAK6+8ApVKhaVLl2LVqlXw9fXFU089hWnTpuHFF18s1f09PT2xdetWfPjhh9iyZQtOnTqFRo0a4euvv0ZaWppZUty8eXP8/PPP+Pjjj7FlyxaIoojHH38c27dvx7Zt28yOVygU+O677zBt2jT89NNPeYttDRo0yGoSDQDjxo1DnTp1sGjRIqxfvx46nQ5169bFjBkzMGbMGAiC+bQ6Wzp48CAOHjxosa59+/Zo06ZNiWKcNGkSNm/ejBMnTmD37t1QKpUICQnBhx9+iBEjRuQNex8xYgR8fHxw7NgxHDx4ELIsIygoCG+88QZeffVV+Pj4lOnzLkxISUnhSj/kFLRaLRISEhAcHAyNjRfAqojc/u//oPn4Y7NyWa1G+v79kBo0KNP7p6QAzz3niYMHrX9WN3WqFm+8kY0yfr23KbZTcgVsp+QK2E4tu3nzpsWVjclxJEmCTqeDWq022yqKXIctf7fYCojKqewJE2AMCzMrF3Q6uE+YUCbbXhXk5wf8/HMGeve23ts8Y4YGkydrYOPdFYiIiIiIygyTaKLySq1G1oIFkC108yoPHYJ65coyD0GjAb79NhPDh1vfF3HZMje8/LIHSrl1IhERERGRXTCJJirHjI88At2YMRbrNNOmQUhKKvMYFApg9mwt3nnHfEXFXL/+qkL//p5ITS3zcIiIiIiISoVJNFE5p33vPUj3tl0oSLh7F+5vvAHIZb8sgiAAU6ZkY86cLIii5fvt369Er15euHbNhSZIExEREVGFwySaqLzz8kLW3LkWq1TbtkH56692C2X4cB1WrcqEm5vlRPr0aQWeesoL//3HlyYiIiIick58p0pUARi6doXOyr6B7m+9BeHOHbvF0ru3ARs3ZsDX13IiHR8vont3Txw/br49FhERERGRozGJJqogtLNmQQoIMCsXb96EZupUu8bStq0R27alo0YNyyuE37olondvT+zaxa3siYiIiMi5MIkmqiBkf39oP/nEYp16zRoo9uyxazxhYRJ27EhHgwaW97fKzBQwZIgHfvhBZde4iIiIiIjuh0k0UQWi798f+qeesljnPnEikJlp13iCg2Vs356BVq0MFusNBgFjxnhgwQK1XeMiIiJyJNkOi34SVSS2/p1iEk1UkQgCsr74ArKXl1mVIi4Omlmz7B6Sv7+MX3/NQPfueqvHvP++O957TwPJ8uhvIiKickOj0UCrtb4tJBGVnFarhUajsdn1mEQTVTBycDC0H3xgsU69aBEUf/9t54gADw9gzZpMPP+8zuoxixa5YcwYd+isH0JEROTyPD09kZ6ejqysLPZIE5WSLMvIyspCeno6PD09bXZdrtpDVAHpRo6EasMGKI8cMSkXJAnu48cj/Y8/AJV95yIrlcCCBVkIDJTwxReWPylcv16N5GQB336bCW9vu4ZHRERkF6IoonLlysjIyEBycrKjwyEAkiTl9WSKIvsgXY1Go0HlypVt+n/HJJqoIhJFZM2fD68OHSDoTYdRK06fhtvChch+/XW7hyUIwPvvZ6NqVRlvv62BLAtmx+zerUJ4uCd+/DETVarwE3oiIip/RFGEt7c3vPmJsVPQarVIS0tDYGCgTYcEk+viRylEFZTUqBGy33jDYp3bJ59A/O8/O0eUb/RoHb7+OgtqteUk+cQJJbp390RcnHmSTURERERUlphEE1Vg2ZMmwfjQQ2blQnY23CdMgCNX8urXT48NGzLg7W05kY6NVeCpp7xw8iRfxoiIiIjIfvjuk6giU6uRNW8eZMG8R1d54ABUq1Y5IKh8HTsasXVrOgIDLSfzN26I6N3bC3v3KuwcGRERERFVVEyiiSo442OPQTdqlMU69w8+gHD1qp0jMtWsmYQdO9JRt67RYv3duwIGDPDEzz/bdyE0IiIiIqqYmEQTEbRTp0IKCjIrF9LS4P7WWw6IyFRoqIwdOzLQsqXBYr1eL2DECHd8+aXazpERERERUUXDJJqIAG9vZM2ZY7FKtWULlL/+aueAzAUEyNi8OQNdu+ot1suygLffdseHH7qB22oSERERUVlhEk1EAADDk09CN2iQxTr3yZOBlBT7BmSBlxewdm0mBg3SWT1mzhwNxo51h95yrk1EREREVCpMookoj3bmTEj+/mbl4vXrcP/gAwdEZE6tBr78Mgvjx2dbPeb779UYOtQDGRl2DIyIiIiIKgQm0USURw4IgPaTTyzWqVetgiI62s4RWSaKwEcfafHRR1lWj9m5U4U+fTxx+zb3kiYiIiIi22ESTUQm9AMHQt+tm8U69wkTgCzriau9jR+vw9KlmVAqLU+CPnpUie7dPREfz0SaiIiIiGyDSTQRmRIEZM2eDdnT06xKcekSNFZ6qh1l0CA91q/PhKen5UT6wgUFunf3wpkzfLkjIiIiotLju0oiMiOHhED7/vsW69QLF0I8ccK+ARWhSxcDNm/OQECAZLH+6lURPXp44cABhZ0jIyIiIqLyhkk0EVmke+UVGB591KxcMBrh8dprgMHyns2O8vDDRuzYkYFatSwn0mlpAiIiPLF5s9LOkRERERFRecIkmogsUyiQNX8+ZJXKvOrUKagXLXJAUPdXt66EnTvT0bSp0WJ9draAF1/0wNdfq+0cGRERERGVF0yiicgqKSwM2a+/brFOM2sWxNhYO0dUtMBAGVu3pqNjR8s95ZIkYNIkd8ya5QbZ8jRqIiIiIiKrmEQT0X1lv/EGjA0bmpULWm3Oat1OmIn6+AA//piBfv10Vo/59FMNXn9dA6PlTmsiIiIiIouYRBPR/bm55QzrFsy3iVLu2wfVd985IKiiubkBK1ZkYdSobKvHrFzphmHDPJxp1y4iIiIicnJMoomoSMbHH4du5EiLde7vvw/h2jU7R1Q8ogh8+qkWH3ygtXrM1q0qRER4IiXFfnERERERketiEk1ExaJ9/31INWualQupqXCfPNkBERWPIACTJmVj0aJMKBSWh54fOqREz55euHLFvLediIiIiKggJtFEVDw+Psj64guLVapNm6DcvNnOAZXM0KF6fP99JtzdLSfSMTEKdO/uhXPn+LJIRERERNbx3WI5sGnTJvTt2xehoaHw8/PD5cuXHR0SlVOGp5+Grn9/i3Xub70FZx8T3b27Ab/+moFKlSzvJZ2YKOLppz1x5IjCzpERERERkatgEl0OZGZmom3btnj33XcdHQpVANpPPoFUqZJZuXjtGjTTp9s/oBJ67DEjtm/PQFCQ5UT6zh0Rffp4Yvt2pZ0jIyIiIiJXwCS6HBgyZAgmT56M9u3bOzoUqgDkKlWgnTnTYp3bypVQ7N9v54hKrmFDCTt3piMszPL+VllZAoYO9cDq1So7R0ZEREREzs5lk+i5c+fCz88Pfn5++Ouvv+x+/3Xr1mHixIno3LkzqlatCj8/P6xZs+a+5xw/fhwDBw5ESEgIatSogW7dumHjxo12ipjIdvRDhkDfpYvFOvcJE+AKe0bVqCEjKiodbdoYLNYbjQLGjfPAF1+4OeNW2ERERETkIC6ZRMfExGDWrFnw9PR0WAwzZszAypUrkZCQgMDAwCKPj46ORvfu3XH48GH069cPw4cPx/Xr1zF8+HAsWLDADhET2ZAgIGv2bMgeHmZViosX4fZ//+eAoErOzw/4+ecM9O6tt3rMRx9pMHmyBkbLndZEREREVMG4XBKt1+sRGRmJpk2bolevXiU694cffkB8fLzVeqPRiIULF0Kn0xV5rQULFuDUqVO4ePEiXn755fseazAYMGHCBIiiiK1bt2LevHn4+OOPsX//ftSrVw8fffSRWVzTp0/P62m39iByJDk0FNr33rNY5zZvHsR//rFzRA/G3R349ttMDB+ebfWYZcvcMGKEO7KtH0JEREREFYTLJdGff/45zp49i4ULF0KhKP4KuleuXMGECRPQu3dvi4m0JEmIjIzE1KlTsXLlyiKv17lzZ4SEhBTr3tHR0bh06RIGDBiAZs2a5ZX7+vpi0qRJ0Ol0WLt2rck548aNw5EjR+77IHI03ZgxMDz8sFm5YDTCffx4wGB5qLSzUSiA2bO1ePttrdVjfvlFjQEDPJGWxr2kiYiIiCoyl0qiT5w4gS+++AJTpkxBo0aNSnRuzZo1sXz5ciQlJSE8PBwJCQl5dZIkYcyYMVi/fj2GDBmCkSNH2jTu/fcWWupiYQ5p165dAQAHDhwwKQ8ICECDBg3u+yByOIUCWfPnQ1aar2StPHEC6iVLHBDUgxEE4O23szFnThZE0fIk6H37lOjXrxKSk7ngGBEREVFF5TJJdHZ2dt4w7gkTJjzQNcLDw7FixQpcuXIFvXv3RkJCQl4P9Pr16zFo0CAsXrwYomjbH8vFixcBAHXr1jWrCwwMhJeXF2JjYx/4+nfu3MGpU6dw4cIFAMC5c+dw6tQp3Llz54GvSVRcUpMmyJ440WKdZuZMiJcu2TegUho+XIdVqzLh5mY5kT5zRoURIxohNpZ7SRMRERFVRC6TRM+cORMXL17EokWLSjSMu7A+ffpg6dKlSExMRHh4OEaMGIF169ahf//+WLJkic0TaABIS0sDAPj4+Fis9/b2zjvmQURFRaFjx4548cUXAQCDBg1Cx44dERUV9cDXJCqJ7DffhLF+fbNyISsL7hMnwtWWt+7d24CNGzPg62s57qQkN/Tu7Y8//uBe0kREREQVjUsk0UeOHMGCBQvw5ptvIiwsrNTXi4iIwJIlSxAXF4eNGzeiV69eWLp0aamSc0caOnQoUlJSzB5Dhw51dGhUUWg0yJo3z2KVcu9eqL7/3s4BlV7btkZs25aOGjUki/W3b4uIiPDAp5+6QbJ8CBERERGVQ06fRBsMBkRGRqJx48Z4/fXXbXJNWZYRHR2d9/3Zs2dx48YNm1zbktweaGu9zXfv3rXaS03kKoxt2yJ7xAiLdZr33oNQhr9jZSUsTMKOHelo0MDy/layLGDWLA0GDvTArVtccIyIiIioInD6JDo9PR0XL17EP//8gypVqphs8ZS7ovWTTz4JPz8/bNmypcjrybKMCRMmYPXq1YiIiMDSpUsRFxeH8PBwXL16tUyeQ+5c6Ny50QVdv34d6enpqFOnTpncm8ietB98AKlGDbNyMSUFmilTHBBR6QUHy9i+PQOtWllfafz331Xo1MkLR4+65mgWIiIiIio+p5/Q5+bmhhdeeMFi3cGDB3Hx4kX06NEDAQEBRW45JcsyJk6ciFWrVqFfv35YtmwZFAoFRFHE6NGjER4eji1btqBatWo2fQ7t2rXD7NmzsXv3bvTv39+k7vfff887hsjl+foi6/PP4fncc2ZV6o0boR84EIaePR0QWOn4+8v49dcMREZ64NdfLa/MnZgookcPT3z8sRavvKKDwI5pIiIionLJ6ZNod3d3LFiwwGJdZGQkLl68iEmTJqFVq1b3vY4sy5g0aRK+/fZbkwQaAAYMGAAAJol0YGCgzZ5Dp06dEBoaig0bNmD06NF5e0WnpqZi9uzZUKvVGDJkiM3uR+RIhp49oevXD+qNG83q3N94A3fbtQN8fR0QWel4eAArV2ZiwQIRH37oCaPRfCCPXi9g8mR3/PmnAvPmZcHLywGBEhEREVGZcvok2lauXr2KLVu2oG/fvli2bBmUhfa1HTBgAGRZxpgxY7Bnzx4MHjz4vtdbtWoVDh06BACIiYkBAHz33Xd5e0K3adMGw4YNAwAolUrMnz8f/fv3R69evRAREQEvLy9s2rQJCQkJ+Oijj1CrVi1bP2Uih9F++imUf/wBMSXFpFy8ehWa//0P2i++cExgpSQIwKhRmahR4wqmTq2Pa9csD9/+6Sc1Tp9W4NtvM9GoEVcdIyIiIipPKkwSXaNGDfz2228ICgoyS6BzDRw4EC1btkS9evWKvN6hQ4fy5mTnOnz4MA4fPpz3fW4SDQAdO3bE9u3bMWvWLGzcuBF6vR5hYWH48MMPERER8YDPisg5yVWrQvvxx/AYO9aszm3FCuj794exbVsHRGYbzZun47ffbmHcOH/s3Wv59eTcOQW6dvXCvHlZGDBAb+cIiYiIiKisCCkpKa61gSuVW1qtFgkJCQgODoZGo3F0OFRasgzPvn2h3LvXrMpYvz7S9+0DXPD/uWA7Vak0mDXLDZ9/fv/nMXJkNj7+WAs3NzsFSRUeX0/JFbCdkqtgW6XCnH51biJyUYKAzHnzILu7m1UpLlyA2+efOyAo21IogKlTs7F+fQb8/KwP216+3A09e3oiPp6rjRERERG5OibRRFRm5NBQaN9912Kd29y5EE+ftnNEZeOppwyIjk5Hy5bWt8E6dkyJTp28sGtXhZlFQ0RERFQuMYkmojKli4yEoUULs3LBYID7a68BRqP9gyoDISE5+0mPGJFt9Zg7d0QMHOiBjz92Ky9Pm4iIiKjCYRJNRGVLqUTW/PmQFeYrWSuPH4f6yy8dEFTZcHMDvvhCi6VLM+HhYXm5CVkW8H//p8GAAR5ITubwbiIiIiJXwySaiMqc1KwZsidMsFin+fhjCHFx9g2ojA0apMfvv6ejfn3r3c1//KFCx45eOHLE8jZZREREROScmEQTkV1kv/UWjHXrmpULmZlwnzQJkMvXRgEPPSRh9+50RETorB6TlCSiZ09PLFmiLm9Pn4iIiKjcYhJNRPbh7o6sefMsVql274bqhx/sHFDZ8/YGVqzIwqefZkGlspwlGwwC3nnHHcOHuyMtzc4BEhEREVGJMYkmIrsxtm+P7JdeslinefddCDdv2jcgOxAEYPRoHaKiMlCzpvVtsH75RY0uXbwQE8OXZSIiIiJnxndrRGRX2unTIVWrZlYu3rkDzdtvOyAi+2jVyojo6HR06aK3esx//ynQrZsX1q1T2TEyIiIiIioJJtFEZF9+fsj6/HOLVeqffoJy+3Y7B2Q/lSvL+PHHTEyZooUgWB7enZkpYPRoD7z+ugZarZ0DJCIiIqIiMYkmIrsz9O4N/TPPWKxzf+MN4O5dO0dkPwoF8M472diwIRP+/taHd3/zjRueftoTcXHcBouIiIjImTCJJiKHyPq//4Ps62tWLl65As3//ueAiOyra1cD9u5Nx6OPGqwec+KEEp07e2HHDqUdIyMiIiKi+2ESTUQOIQcGIuujjyzWqZcvh+LwYTtHZH/BwTKiojLwyivZVo9JSRExeLAnPvrIDUbr204TERERkZ0wiSYih9G/8AIMHTqYlQuyDPcJE4Bs68lleaFWA//3f1qsWJEJT0/rm0V/8YUG/fp54sYNDu8mIiIiciQm0UTkOIKArHnzIGs0ZlWKc+fg9sUXDgjKMfr312P37nQ0bGi9uzk6WolOnbxw6JDCjpERERERUUFMoonIoaQ6daB95x2LdW5z5kCMibFzRI7TsKGE339Px8CBOqvHXL0qondvTyxYoIZsveOaiIiIiMoIk2gicjjd2LEwNmtmVi7o9XB/7TVUpMnAXl7A0qVZ+PzzLKhUlrNko1HA+++7Y9gwD6Sm2jlAIiIiogqOSTQROZ5Sicz58yErzIcpK48ehXrZMgcE5TiCAIwcqcP27RkICrK+DdbmzSo88YQXTp/mSzkRERGRvZT5O6+UlBTExMQguwIsEERED05q0QK6ceMs1mk++ghCfLydI3K8Rx4xIjo6HU8+qbd6TGysAt26eWHNGpUdIyMiIiKquEqdRJ88eRIff/wxdu/ebVKelZWFESNGoE6dOmjfvj0aNWqEX3/9tbS3I6JyTPv22zDWrm1WLmRkwP3111ERJwH7+8tYty4T772nhSBYfv5arYCxYz3w2mvuyMqyc4BEREREFUypk+jVq1fjiy++gFzoze3MmTPx888/Q5ZlyLKMlJQUvPLKK4ipQIsEEVEJubsja948i1Wq33+H6scf7RyQcxBF4K23srFxYwYqV7Y+vHvVKjW6d/dCXBy3wSIiIiIqK6VOog8ePAiNRoMnnngir0yn0+Hbb7+FSqXC+vXrERcXh9GjR0Ov1+PLL78s7S2JqBwzduwI3bBhFus0b78NITnZzhE5j86dc4Z3P/aYweoxp04p0LGjN6KilHaMjIiIiKjiKHUSfePGDVSvXh2imH+pI0eO4O7du+jRoweefPJJ+Pr6Ytq0afD09MSBAwdKe0siKuey/vc/SIGBZuXi7dvQvPuuAyJyHjVryti6NQORkdbXmUhLE/Dcc56YPt0NBuv5NhERERE9gFIn0SkpKahUqZJJ2ZEjRyAIArp27ZpX5u7ujtDQUCQlJZX2lkRU3vn5IeuzzyxWqdevh/K33+wckHNRqYBZs7RYuTIDXl7W54nPnatBnz6euH6dw7uJiIiIbKXUSbS7uzuSCw2vPHToEADg8ccfNylXq9UmPdZERNYY+vSBvndvi3Xur78O3L1r54icT9++BvzxRzoeesj6PtoHDijRsaMX9u833z6MiIiIiEqu1BltgwYNEB8fj3///RcAcOvWLezbtw+VK1dGw4YNTY69evUqAgICSntLIqogsv7v/yD7+JiVi4mJ0MyY4YCInE/9+hJ27UrH4ME6q8dcvy6iTx9PzJunrogLnBMRERHZVKmT6L59+0KWZQwcOBDvvfcewsPDodPpEBERYXJcQkICrl27hjp16pT2lkRUQcjVqyPro48s1qmXLoXiyBE7R+ScPD2BL7/Mwty5mVCrLWfJRqOAadPcMXSoB1JS7BsfERERUXlS6iR61KhRaNu2La5cuYLFixfj33//Rb169TBlyhST4zZu3AgA6NChQ2lvSUQViP6FF2Bo186sXJBluL/2GqCz3gNbkQgC8NJLeuzcmY6QEOvbYEVFqdC5sxdOnuTUGiIiIqIHUep3UWq1Gps3b8bq1asxbdo0LF++HNHR0fD39zc5TqFQYMyYMejTp09pb0lEFYkoImvePMhubmZVirNn4TZnjgOCcl4tWkjYuzcd3bvrrR4TF6fAU095YdUqlR0jIyIiIiofhJSUFM6QI6eg1WqRkJCA4OBgaDQaR4dDTsZtzhxoPvzQrFxWqZC+bx+kRo3sEoertFNJAubOdcOMGW6QJOurcw8dqsP//V8WPDzsGByVOVdpp1SxsZ2Sq2BbpcI4no+IXEL2uHEwNmliVi7o9TnDuiXrQ5grIlEEJk3KxsaNGahSxfrPZs0aNZ580gsXL/LPAREREVFxlPpdU3JyMvbu3Yv//vvPrO6bb75Bu3btUKdOHQwcOBAXLlwo7e2IqKJSqZC1YAFkC9vkKY8cgXr5cgcE5fw6dTIiOjodbdoYrB5z5owCTzzhhU2blHaMjIiIiMg1lTqJ/vLLL9GvXz/89ddfJuUrV67EG2+8gZiYGNy5cwe7du1CeHg4bt++XdpbUiGbNm1C3759ERoaCj8/P1y+fNnRIRGVCWPLltCNHWuxTvO//0FISLBzRK6henUZmzZlYNy4bKvHpKUJGDbME1OnaqC3Pp2aiIiIqMIrdRK9b98+KBQKhIeHm5TPnj0bADB+/HisXr0abdq0wY0bN7B48eLS3pIKyczMRNu2bfHuu+86OhSiMqd95x0YQ0PNyoX0dLi/8Qa4EbJlKhUwY4YWq1ZlwMfH+s9o4UI3PPOMJ65etT6PmoiIiKgiK3USnZCQgMDAQHh5eeWV/fPPP0hISMDjjz+O//3vf+jVqxe++eYbKBQK7Nixo7S3pEKGDBmCyZMno3379o4OhajseXgga948i1WqnTuh+uknOwfkWp55xoA//khH48ZGq8ccOqREx45eiI5W2DEyIiIiItdQ6iT69u3bqFatmknZ4cOHAQA9e/bMKwsMDESdOnUQFxdX4ntotVq8++676NGjBxo1aoTAwEA0aNAA3bt3x+rVq6F3wNjDdevWYeLEiejcuTOqVq0KPz8/rFmz5r7nHD9+HAMHDkRISAhq1KiBbt265e2fTUTFZ+zUCbqhQy3WaaZMgXDrlp0jci1160rYtSsdQ4da32P75k0Rfft6YvZsN67ZRkRERFRAqZNoURSRnp5uUnbkyBEIgoDWrVublPv4+ECns/6mzZqMjAx8/fXXEAQBTz31FMaOHYvevXsjKSkJ48aNw+DBgyHZ+V3ejBkzsHLlyrye+KJER0eje/fuOHz4MPr164fhw4fj+vXrGD58OBYsWGCHiInKF+2MGZCqVjUrF2/dgoZTG4rk7g4sWpSF+fMz4eZmeXi3JAn43/80ePZZD6Sk2Dc+IiIiImdV6iQ6JCQEsbGxuHPnDgBAr9dj9+7dcHd3R8uWLU2OvXXrFipXrlzie1SqVAnx8fGIiorC/Pnz8cEHH2D27Nn4+++/0b59e+zevRu//fZbkdf54YcfEB8fb7XeaDRi4cKFxUr0FyxYgFOnTuHixYt4+eWX73uswWDAhAkTIIoitm7dinnz5uHjjz/G/v37Ua9ePXz00UdmcU2fPh1+fn73fRBVZHKlSsj67DOLdep166D8/Xc7R+Sahg3TY+fOdISGWh/evWOHCh07euPECW6DRURERFTqd0RdunSBXq/HiBEjsG3bNowfPx63b99G165doVTmb5eSmpqKuLg41KxZs+RBiiLUarVZuVKpRO/evQEAsbGx973GlStXMGHCBPTu3dtiIi1JEiIjIzF16lSsXLmyyJg6d+6MkJCQYsUfHR2NS5cuYcCAAWjWrFleua+vLyZNmgSdToe1a9eanDNu3DgcOXLkvg+iis7Qpw/0PXpYrHOfOBEoNEqGLGveXMKePeno2dP61Jj4eBFPPeWFlStVXLuNiIiIKrRSJ9ETJ05EYGAg/vjjDwwdOhTr1q2DRqPBlClTTI7bvn07ZFlGmzZtSnvLPJIk4fd7vU1hYWH3PbZmzZpYvnw5kpKSEB4ejoQCW+FIkoQxY8Zg/fr1GDJkCEaOHGmzGAFg//79AHI+cCisa9euAIADBw6YlAcEBKBBgwb3fRBVeIKArC++gOzjY1YlJiRA8/HHDgjKNfn5AWvWZOLDD7OgUFjOknU6ARMnemDMGHdkZNg3PiIiIiJnUeokumrVqti9ezdGjx6NLl264KWXXsKePXvQuHFjk+MOHTqEJk2aoHv37g98L51Oh1mzZmHmzJl466230Lp1a/z2228YOnQoOnXqVOT54eHhWLFiBa5cuYLevXsjISEhrwd6/fr1GDRoEBYvXgxRtO2QxYsXLwIA6tata1aXu7J5UT3p93Pnzh2cOnUKFy5cAACcO3cOp06dyhtiT1SeyTVqQDt9usU69ZdfQnH0qH0DcmGCAEyYoMOvv2YgMND6OhPr1qnx5JNeuHCBw7uJiIio4lEWfUjRatSogU8++eS+x8ydO7fU99HpdPj000/zvhcEAePHj8e0adOKfY0+ffrAaDTilVdeQXh4OFq2bImNGzeif//+WLJkic0TaABIS0sDkLOwmiXe3t55xzyIqKgojB07Nu/7QYMGAQAWLVqEoVZWMCYqT3QvvQTVjz9CeeiQSbkgy3B/7TWk79kDWJgSQpa1b2/E3r3pePllDxw8aPnPREyMAl26eGHhwkz06WOwc4REREREjuNS3QheXl5ISUnB7du3cebMGXz++edYtWoVevfuXaIkNCIiAkuWLEFcXBw2btyIXr16YenSpVAoXHNP1KFDhyIlJcXswQSaKgxRRNb8+ZAtJMqKmBi4WdlXmqyrVk3Gpk0ZmDhRa/WYu3cFvPiiJ955RwMH7DRIRERE5BA2TaJv3LiBdevWYcaMGZgyZQpmzJiB9evX4+bNm7a8DURRRM2aNTFixAjMmzcPhw8fxhdffFHs82VZRnR0dN73Z8+exY0bN2waY0G5PdDWEv27d+9a7aUmouKR6tdH9uTJFuvc/u//IJ4/b+eIXJ9SCUyfno3vv8+Aj4/11cSWLHFD796euHJFsGN0RERERI5hkyQ6OzsbkydPRtOmTREZGYnZs2dj2bJlmD17NsaMGYMmTZrg7bffRnZ2ti1uZ+KJJ54AkL94V1FkWcaECROwevVqREREYOnSpYiLi0N4eDiuXr1q8/iA/LnQuXOjC7p+/TrS09NRp06dMrk3UUWSPWECjBYWGRR0OrhPmADYeT/58qJnTwP27k1H06bWt8H6808lOnb0wp49rjmih4iIiKi4Sp1ES5KEZ599FsuXL4dOp0NAQADatGmDfv36oU2bNggICIBOp8PSpUsxdOhQyDbeG+XatWsAAJVKVeSxsixj4sSJWLVqFfr164dly5Zh0KBB+Oqrr3Dp0iWEh4fnXc+W2rVrBwDYvXu3WV3u6uK5xxBRKahUyFqwALKFtQ2Uhw5B/c03DgiqfKhdW8LOnekYNkxn9Zhbt0T06+eJzz5z4+cVREREVG6VOolevXo1/vjjD3h7e2P+/PmIiYnB1q1bsWLFCmzduhX//vsvFixYAF9fX+zevRtr1qwp8T3Onj2LzMxMs/LMzEy89957AIAnn3zyvteQZRmTJk3Ct99+m5dA586BHjBggEkiff369RLHeD+dOnVCaGgoNmzYgFOnTuWVp6amYvbs2VCr1RgyZIhN70lUURkfeQS6MWMs1mmmT4dw5YqdIyo/3N2B+fOzsGhRJjQayx+IyrKAmTM1GDzYA7dvc3g3ERERlT+lXp173bp1EAQBq1atsrjNlEKhwPPPP4/g4GD07dsXa9euxfPPP1+ie2zcuBGLFy9G69atERISAm9vbyQlJWHXrl24ffs22rRpg1dfffW+17h69Sq2bNmCvn37YtmyZVAqTZ/6gAEDIMsyxowZgz179mDw4MH3vd6qVatw6N5KwDExMQCA7777Lm9YeZs2bTBs2DAAgFKpxPz589G/f3/06tULERER8PLywqZNm5CQkICPPvoItWrVKtHPhIis0773HlRbtkCMjzcpF+7ehfsbbyBz7dqc/ZzogQwdqkezZka8+KIHYmMtD9/+7TcVOnZU4NtvM/HII9aHgRMRERG5GiElJaVU46tDQ0NRqVIl/P3330Ue27JlS9y5cwdxcXElusfff/+NlStX4siRI0hKSkJGRgZ8fHzQuHFj9O/fH88//7xZUmxJXFwcgoKC7nvsf//9h3r16hV5rcjISKxdu9Zq/bPPPoslS5aYlB07dgyzZs3CkSNHoNfrERYWhrFjxyIiIqLI+1UEWq0WCQkJCA4OhkajcXQ45OKUu3fD08rvVuY330Dfr98DXZftNF9qKjBunAc2b7Y+nUalkjFrlhYjRuj4uYUdsZ2SK2A7JVfBtkqFlTqJDgwMROPGjS3O9y2sS5cuiImJKZN5x+T6+AJFtuYeGQm1hQ+7pCpVkP7nn5D9/Ut8TbZTU7IMLFqkxrRpGhiN1rPkgQN1mDMnC15edgyuAmM7JVfAdkqugm2VCiv1nOjAwEBcuHABWVlZ9z0uMzMTFy5cQNWqVUt7SyKiYtF+/DGkgACzcvHmTWimTnVAROWPIADjxumwZUsGqlWzvprYjz+q0a2bF86ds+nOikRERER2V+p3Mx06dEBGRgbefffd+x737rvvIiMjAx07diztLYmIikX294f2008t1qm//x6KPXvsGk951qaNEdHR6ejQwWD1mLNnFejSxQs//1z0bgpEREREzqrUSfSECROgUqnw7bffon379li9ejWOHz+OxMREHD9+HKtXr0a7du2watUqqNVqvPbaa7aIm4ioWPQREdB3726xzmPCBCAjw84RlV9Vq8r45ZcMvPGG1uoxGRkCXn7ZA5Mna6CzvlsWERERkdMq9ZxoIGf17FdffRVarRaChZVjZFmGRqPBkiVL0Ldv39LejsopzjehsiIkJsK7TRsId++a1WWPGwftjBnFvhbbafFs367E6NEeSE21Pk/60UcN+OabTAQHl/rPEBXCdkqugO2UXAXbKhVmk8lp/fr1Q3R0NIYOHYqqVatCluW8R9WqVfHCCy8gOjqaCTQROYQcFATttGkW69SLF0Nx/LidIyr/nn7agL1776J5c+vbWx09qkSnTl74/fdS77ZIREREZDc26YkuLC0tDenp6fDy8oKPj09eeadOnZCamooTJ07Y+pZUDvBTPipTkgTPHj2g/PNPsypj48ZI37MHUBU9V5fttGS0WuDddzX4+ms3q8cIgozJk7MxeXI2FJa3naYSYjslV8B2Sq6CbZUKK5NlUn18fFCjRg2TBBoAEhMTER8fXxa3JCK6P1FE1vz5kNVqsyrFmTNwW7DAAUGVfxoNMHu2Fl9+mQl3d8uf2cqygE8/1aBDBy98+qkbTp0SIXOENxERETkp7jVCRBWG1LAhst9802Kd26efQrxwwc4RVRxDhujx++/pqFfP+vDumBgFZs3SoGNHbzRt6o233tJgzx4FFyAjIiIip8IkmogqlOyJE2EMCzMrF7Kz4T5hAiBZ3+uYSicsTMLu3eno00df5LGJiSKWLXND375eqFfPByNGuGPDBhVSUso+TiIiIqL7YRJNRBWLWp0zrNvCTgLKgwehWrXKAUFVHD4+wMqVmZg1KwtKZfHGbKelCfjpJzVGjvRAvXo+6NvXA199pUZ8vPWVv4mIiIjKCpNoIqpwjI8+Ct3o0Rbr3D/4AEJSkp0jqlgEAYiM1CEqKgNBQSXr+TcYBOzZo8KUKe5o1swH7dt7YeZMN5w4wXnUREREZB9MoomoQtJOnQopONisXEhLg/ubb4IZWdl77DEjjhy5i3nzMvHUU3q4uZX8Z376tAKffaZB587eaNLEG2++qcHvvyuRnV0GARMRERGBSTQRVVReXsiaM8dilSoqCspNm+wcUMXk4QG8+KIe69dn4uLFNKxalYFnn9XB37/kc9OvXBGxfLkb+vf3RL16PnjpJXesX8951ERERGRbypKe8Omnnz7wzbKysh74XCIiWzN06wbdoEFQr19vVuf+1lu426kT4Odn/8AqKC8v4JlnDHjmGQMMBuDIEQWiolSIilIiNrZkG0jfvSvgl1/U+OUXNRQKGW3bGtGjhx49e+oRGspRBkRERPTghJSUlBK9m6hUqRIECwvyFIcsyxAEAbdv336g86l840b25AjCrVvweuwxiLdumdXpXngBWYX2j2Y7tT9ZBs6fFxEVpcK2bUr89ZcCsvzgi4qFhRnRs6cePXsa0KKFEWI5HJPFdkqugO2UXAXbKhVW4p7otm3bPnASTUTkbOTKlaH95BN4vPKKWZ36u++gGzAAxk6dHBAZ5RIEoGFDCQ0bZuP117Nx/bqAHTuUiIpSYc8eJbTakv1NiolRICZGgc8/B6pXl/D003r06GFAx44G8L0RERERFaXEPdFEZYWf8pHDyDI8Bg2C6rffzKqMtWsj/cCBnMm7YDt1NhkZwJ49OQn1jh1KJCc/eLeyp6eMLl0M6NlTj+7dDfD3d90/j2yn5ArYTslVsK1SYeVwEBsRUQkJArJmz4bs6WlWpbh0CZpPPnFAUFQcnp5Ar14GLFqUhXPn7mL79nRMmJCN+vWNJb5WRoaAzZtViIz0QL163ujZ0xMLF6oRG8s/lURERJSP7wyIiADIwcHQfvCBxTr1woUQT5ywb0BUYgoF0Lq1ER9+qMVff6Xjr7/u4n//y0Lr1gYIQsl6lSVJwMGDSkyd6o6HH/ZG69Ze+PBDN/z1lwJSyRcOJyIionKESTQR0T26kSNhaNXKrFyQJHiMHw/o9Q6Iih5U/foSXntNh+3bM3Dhwl0sXJiJXr30cHcv+TDts2cVmDNHgyef9EKjRt547TV3bN+uBDedICIiqniYRBMR5VIokDV/PmSVyrzqn3+gXrTIAUGRLQQEyHj+eT3WrMlEbGwafvghA8OG6VC1asm7lW/cELFqlRpDhniibl0fDB3qgdWrVUhO5qKbREREFQGTaCKiAqSHHkL2pEkW6zSffAJFbKydIyJbc3cHnn7agPnzs3D27F389ls6Xn9di0aNSj6POjNTwNatKowb54H69b3x9NOemD9fjQsX+OeViIiovOJfeSKiQrInTYKxUSOzckGrhc+bb+ZsXEzlgigCrVoZMW1aNg4fTsfx43cxY0YW2rY1QBRL9v8sywIOH1bigw/c0aqVN1q18sK0aRocPqyAseT5ORERETkpJtFERIW5ueUM6xbMh+eqDx5EwK+/OiAosoc6dSSMG6dDVFQG/vvvLpYsyUR4uB6eniX/4OTCBQXmzXPD0097oWFDb4wd646tW5XIzCyDwImIiMhumEQTEVlgfOwx6EaOtFgXNG8exOvX7RwR2Zu/v4xnn9Xju+8ycfFiGtavz8Dw4dmoVq3k86iTk0WsWaPG0KGeqFPHB0OGeGDVKhVu3OA8aiIiIlcjpKSkcFwiOQVuZE9O5+5deLdpAzEx0axK2707sr/9FmBbrXAkCThxQoGoKCWiolSIiVE88LUEQUarVkb06GFAz556NGggwcIAiBLj6ym5ArZTchVsq1QYk2hyGnyBImek3LkTnoMGWayTAgKgGzkSupEjIQcE2DkychZxcQKiolTYtk2FgwcVMBofPAuuU8eInj0N6NFDj8cfN0KpfLDr8PWUXAHbKbkKtlUqjEk0OQ2+QJGzcn/lFah//NFqvazRQDdkCHSvvgqpQQM7RkbO5s4dAb/9pkRUlBK7dqmQnv7gCbW/v4Tu3XMS6i5dDPDyKv65fD0lV8B2Sq6CbZUKYxJNToMvUOSshORkeD32GMTbt4s8Vt+9O7LHjYOxfXvYZFwuuazsbGD/fiW2bVNi2zYVrlx58GVI3NxkdOqUM+T76acNqFbt/n+6+XpKroDtlFwF2yoVxiSanAZfoMiZKbdsgcfw4RD0+mIdb2zeHNnjxkHfty+gUpVtcOT0ZBk4eVJEVJQKUVEqnD794POoAeCRRwx586gfesh8HjVfT8kVsJ2Sq2BbpcKYRJPT4AsUOTvFgQNwe+cdqE6dKvY5UlAQskePhm7YMMDXtwyjI1cSHy9g2zYVtm1TYv9+JQyGBx+1EBqavzBZmzY586j5ekqugO2UXAXbKhXGJJqcBl+gyBVos7Jw59dfUfvnn+G2c2exz5O9vaF74QVkjxkDOSSkDCMkV5OSAuzalZNQ//abCmlpD55Q+/lJeOopA7p1y0SDBrFo1KgmX0/JafHvPrkKtlUqjEk0OQ2+QJErKNhOPRISoF68GOq1ayFotcU6X1YooO/TB7px42B8+OEyjpZcjU4HHDyowNatOat9JyY++DxqhUJC7doSateWERoq5T1q15ZQq5YET08bBk70APh3n1wF2yoVxiSanAZfoMgVWGqnQnIy1MuXQ718OcTk5GJfy9CmDbLHjYOhRw9AfPBkiconWQb++UfEtm0586hPnizdPOrCAgMls+Q69+vAQJnr4lGZ4999chVsq1QYk2hyGnyBIldw33aalQXV+vVwW7QIivPni31NY9260I0dC92QIYCHh40jpvIiMVHA9u0qREUpsW+fEnp92WW57u7mvde5/4aESHBzK7NbUwXCv/vkKthWqTAm0eQ0+AJFrqBY7VSSoPztN7gtXAjlvn3Fvrbk7w/diBHQvfIK5KpVbRQxlUdpacDvv+fMo96xQ4XUVPt1GwuCjBo1ZIsJdmioBH9/9mJT8fDvPrkKtlUqjEk0OQ2+QJErKGk7FU+cgNvixVD9/DMEg6FY95Dd3KAfNAjZY8dCatSotCFTOafXA4cOKfK2z4qPd+zUAB8fGbVqWU6wg4Ik7vhGefh3n1wF2yoVxiSanAZfoMgVPGg7Fa5cgdtXX0G9ciWEtLRin6d/8klkjxsHY8eOYPceFUWWgZgYEVFRwIkT2UhO9sHly0pcu+Ycc+4VChlBQfK9xNpostBZ7doSd4GrYPh3n1wF2yoVxiSanAZfoMgVlLqd3r0L9XffwW3JEogJCcU+zdikCbLHjYM+IgJQq0t+X6pQCrfTzEzg8mURcXEiLl3K+Tf3cfmyiOxs5/iAplIl80XOcr+vUUOGwrZrq5GD8e8+uQq2VSqMSTQ5Db5AkSuwWTs1GKDavBnqhQuhPHas2KdJ1asje/Ro6F56CfDze/D7U7lWknYqScDVq4LFBDsuTkRysnP0YqvVMkJCTJPrgj3ZXl6OjpBKin/3yVWwrVJhTKLJafAFilyBzdupLENx+HDOImRRURDk4r0ky56e0D3/PLIjIyGHhpY+DipXbNlO09JyerEvXRLz/s1NuBMSRBgMztGLXbWq5QQ7NFRCtWpc7MwZ8e8+uQq2VSpM6egAiIgqNEGAsU0bZLZpA/HiRaiXLIF6zRoIWVn3Py0jI2eO9bJlMISH58ybbtXKTkFTReLjAzRtKqFpU8mszmAArlwR8nqt8xNsBS5dEpGWZr/M9cYNETduiDhyxLzO3T1/sbPCCXatWhL4npiIiEqCPdHkNPgpH7kCe7RT4fZtqL/+GuqlSyHeuFHs8wyPP47ssWNh6NULnDxasTnL62lKCnDpkqJQgp3z75UrAiTJObqHa9QwT65zv65cmb3YZcVZ2ilRUdhWqTAm0eQ0+AJFrsCu7TQ7G6off4TbokVQ/PtvsU8z1q4NXWQkdEOHAp6eZRggOStXeD3V6YCEhMI92PlzsTMynCNz9fY27cUOCpJQo4aEmjVl1KghoWpVLnj2oFyhnRIBbKtkjkk0OQ2+QJErcEg7lWUod++GeuFCqP74o9inSX5+0L38MnSjRkGuVq0MAyRn4+qvp7IMJCcLVhPsq1edY7EzIGfbrurVZdSsmZNc16gh30uyc76uWVNCYKAMJSfQmXH1dkoVB9sqFcaXdCIiZycIMHTtCkPXrhD/+QduixZB9dNPEPT6+54mpqRAM3s23BYsgH7AAGSPGwepcWM7BU304AQBqFJFRpUqRrRqZTSrz8oC4uMtJ9iXL4vQau3Xi200CkhMFJCYaD2xF0UZ1arJZkl2bm92jRoSqleXoVLZLWwiIioF9kST0+CnfOQKnKWdClevQr10Kdy+/hpCamqxz9M/8QR048bB0KULONGz/HKWduoIkgRcvy6YJNgFVxW/edN5erELEgQZVavKeT3YhXuzcxNtNzdHR2o7FbmdkmthW6XCmEST0+ALFLkCp2un6elQr1kD9ZIlUMTFFfs0Y1gYsl99FfqBA1Gu3pUTACdsp07k7t2cLbssJdjx8SL0euf+cKlKlfwkO2d+dn5vds2aMqpXl+Du7ugoi4ftlFwF2yoVxiSanAZfoMgVOG07NRqh3LIFbosWQWlpjx8rpMBA6EaNgu7llyFXqlSGAZI9OW07dXJGo+mWXYXnZKekOGcvdmGVK5v3ZhccQl69uuQUaw6ynZKrYFulwphEk9PgCxS5Aldop4ojR+C2cCGUW7ZAkMz39rVE9vCAbuhQ6CIjIdWpU8YRUllzhXbqiu7eBZKSRCQl5WzRlfO1gCtXRFy5kvO1qyTafn75Q8VNE+38hNvLq2xjYDslV8G2SoVxYTEionLG+NhjyFy1CkJcHNwWL4Z6zRoIGRn3PUfIzITbsmVQL18OQ69eyB43DsbHH+e8aaICvL2Bhg0lNGxo/cOpjAzg6tWcJDsnsc5PtHO/vnXL8Yl2SoqIlBQgJsb6/lw+PpZXHS+4IJqPD18miKjiYU90ObBp0yZ8/fXXOHHiBFJSUnDy5EnUqlXL0WGVGD/lI1fgku00JQVu33wD9VdfQbx2rdinGR59FNnjxsHQuze4P49rccl2WoFkZeUn2gV7tgsm2s66AFphXl6Wh40X7OX29bWcaLOdmpIkwGDImVaQ+5AkAUajabkk5awKn/t9bl3usQAQGJjzYQf3MLcNtlUqjO+KyoHMzEy0bdsWPXv2xOTJkx0dDhE5Gz8/ZL/+OrLHjoXqp5/gtnAhFGfOFHma8uhRKF96CVJICLIjI6F7/vmcrjgiKhV3d6BOHQk5MyfMt/ACgOxs4OpVy73Zucn3jRsCZNmx3cDp6QLOn1fg/Hnr2ZqHh+n2XrkLogUESJBlD9y+rYRSqTBJCgsnivkP87LcY4tKNHPKTMvzE1eh0Lm55wlmZfeLK/e4/FiEQufmXLNwmdFo+/9HpVJGcLCEWrUk1Kolo1YtCaGhud9LqFxZ5igCogfEnuhyJCYmBm3btmVPNFEZKhftVJah2LsXbgsXQrVrV/FP8/GBbvhwZI8aBblmzTIMkEqrXLRTKpJOB1y7Zr03OylJxLVrAiSJmRKZ8/KSERKSn1QXTLBr1XKOxeecBV9TqTCX6IlOSkrCL7/8gt9++w0XLlzA9evXUalSJTz++OOYMGECHn30UbvHtG7dOhw6dAgnTpxATEwMdDodFi1ahKFDh1o95/jx45g1axb+/PNPGAwGhIWFYezYsejXr58dIyeiCk8QYOzcGZmdO0P891+4LVoE1fr1EHS6+5+Wlga3efOgXrQI+ogIZI8bB6lZMzsFTUSFqdVASIiMkBAjrPVoGwwFE23z3uykJBFXrwpl0hNKzi09XUBMjMLqvPgqVawn2EFBMmf5UIXmEs1/6dKlmDt3LmrXro0nnngCAQEBuHjxIrZu3YqtW7di+fLliIiIsGtMM2bMQEJCAipXrozAwEAkJCTc9/jo6Gj0798fGo0GERER8PLywqZNmzB8+HAkJiZi/PjxdoqciCif9NBDyFq4ENr334d62TKoV6yAeOfOfc8RDAao16+Hev16GDp2zJk33a0bILrGHE6iikSpBIKCZAQFWU6ygZzhxDdu5CTViYn5yXVub/aVKzmJtrPvoU22dfOmiJs3RRw9al6nUMioWdN8iHju91WqcKg4lW8uMZx706ZN8Pf3R/v27U3KDx48iD59+sDT0xPnzp2Dm5vbfa/zww8/oG3btggJCbFYbzQasWTJEowaNQpqtfq+19qzZw/q1KmDkJAQzJkzBx9++KHVnmiDwYBWrVohKSkJv/32G5rd67lJTU1F165dER8fj6NHj5rENX36dMydO/e+MaSkpJh8z+HcRGWv3LfTjAyo166FevFiKGJji32asWFDZI8dC/2gQUB5/Lm4mHLfTsnuJAm4eVMw6cXO3dYr99+kJBE6HTMnypkHX3CoeOEebVdbXoOvqVSYS/REP/PMMxbL27Ztiw4dOmD37t2IiYlBy5YtrV7jypUrmDBhAgIDA7FlyxazRFqSJERGRmL9+vVQq9UYNWrUfWPq3LlzseOPjo7GpUuXMHTo0LwEGgB8fX0xadIkvPrqq1i7di2mTJmSVzdu3Dg899xzxb4HEZFNeHpCN3IkdMOHQ7ltG9wWLYLy0KEiT1OcOweP116D9NFHOeePHAm5cmU7BExE9iCKQGCgjMBAI6y93ZJl4NYtwaw3u/AQcq3WORJtQchZvdr0YV4mijnlSmXB7y0fW7A897ic8+RC5+bWyWZllq9r6Zo55YXP1+uBhAQRly+LiIvL+ffKFfvOjc/MFHD2rAJnz1oeKu7vX3iYuJz3dVCQhCL6sogcziWS6PtRqVQAAEURa/jXrFkTy5cvx/DhwxEeHo4tW7YgODgYQE4CPWbMGKxfvx5DhgzByJEjbRrj/v37AQBdunQxq+vatSsA4MCBAyblAQEBCAgIsGkcRETFplDA0Ls3DL17Q3HsGNQLF0L1668QJOv74wKAePMmNLNmwW3uXOiefRa6V1+FVK+enYImIkcSBCAgQEZAgIwWLSy/VsgycOeOgCtXBMTFGfDffykICPCHu7vqXpIoF0oyLSeK1hNeuVCSaX5+bl1FGm6s1+PezzwnqS6YYF++LCI52b7TcW7fFnH7toi//zavE0UZNWrIFnuwa9WSEBgoc/YQOZxLJ9EJCQnYs2cPqlWrhsaNGxd5fHh4OFasWIERI0agd+/e2LJlC2rWrJnXAz1o0CAsXrwYoo1/My9evAgAqFu3rlldYGAgvLy8EFuCYZOF3blzBwkJCbh06RIA4Ny5c0hNTUVwcDAqVar0wNclIgIA4yOPIOubb6C9fBluX34J9XffQUhPv+85QlYW3L7+GupvvoHh6aeRPXYsjO3aVax3rURkRhAAf38Z/v4y6tfXoVGjWwgO9uAskDKmUgGhoTJCQy0vQpeejryEumCCHR+f829Ghv1euyUpZzRDYqKIQn1MAACNxvJQ8dyHn5/dQqUKzGWTaL1ej9GjRyM7OxvTp08vsic6V58+fWA0GvHKK68gPDwcLVu2xMaNG9G/f38sWbLE5gk0AKSlpQEAfHx8LNZ7e3vnHfMgoqKiMHbs2LzvBw0aBABFrhZORFQScq1a0M6aBe2UKVCvWgW3L7+EmJR033MEWYZq2zaotm2DoWVL6MaOhb5Pn5x3dERE5BS8vIDGjSU0bmw+gkCWgeRkwWKSffmygIQE0a6ru2u199+b3M/P8r7YtWpJCAmRUMQSSkTF4pJJtCRJePXVV3Hw4EG8+OKLGDJkSInOj4iIgMFgwKhRoxAXF4devXph6dKlxU7Enc3QoUOZLBOR/fj5Qffaa9BFRkK1cSPcFi6E4tSpIk9T/v03lCNHQpo+HdljxkA3bBhg5cNFIiJyDoIAVKkio0oVIx591LwX22DIGSpeMMku+Lh+3b5jr1NSRKSkACdPmr+vFwQZ1avLFnuwQ0MlVK/OoeJUPC6XREuShLFjx+LHH3/EoEGDMGfOnBJfQ5ZlREdH531/9uxZ3LhxA9WrV7dlqHlye6Ct9TbfvXsXfhx7QkSuRqWCftAg6AcOhGLfvpz9pnfsKPI0MTER7lOnQvPZZ9ANG4bs0aMh31ujgoiIXItSiXs9v5aHimdmIm9YeMF52LmPu3ft14sty0LeSvKW1sxUq2UEB5vPw65eXQmVyjU726hsuFQSndsD/cMPP2DAgAEPNPxalmVMmDABq1evRkREBJ5++mlERkYiPDwcmzdvLpNEOncu9MWLF9GiRQuTuuvXryM9PR0PP/ywze9LRGQXggBjx47I7NgR4rlzcFu8GKoffoCQnX3/09LS4LZwIdRLlkDfrx8MvXrB8MgjOQk1504TEZULHh5Ao0YSGjWyPFT8zh2hQIItmAwZT0gQ7bo/uU4n4OJFBS5eLJwwewKoDH9/CXXrSqhdO+ffOnVy/zVyLnYF4zJJdMEEOiIiAl999VWJh1/LsoyJEydi1apV6NevH5YtWwaFQgFRFDF69Oi8VburVatm09jbtWuH2bNnY/fu3ejfv79J3e+//553DBGRq5MaNkTWvHnQTp0K9fLlUC9fDvHWrfueIxiNUG/YAPWGDTnXqFoVxkcegfHRR2F49FEYW7bksG8ionIof6E5I1q2NO/FNhqBq1cFq73YV686ZlXxv/4yr/P3z0mqcx/5STYT7PLIJZLo3CHcP/zwA/r27ftA85dlWcakSZPw7bffmiTQADBgwAAAMEmkAwMDbRZ/p06dEBoaig0bNmD06NF5e0WnpqZi9uzZUKvVJZ7XTUTkzOQqVZD9zjvInjgRqnXr4LZoERQXLhTrXPHGDYj3FiMDAFkQIDVokJ9YP/IIpLAwLk5GRFTOKRRAUJCMoCAj2rUzT7K12pw9sa1t3ZWaar9e7NwE++hR87pKlfKTaibY5YOQkpIiOzqIosyaNQuffvopvLy8MGbMGIsJdK9evfKSU0uSkpLQqVMntGvXDsuXL4dSaf75wY8//ogxY8Zg8eLFGDx48H1jWrVqFQ7dm0wRExODkydPonXr1qhduzYAoE2bNhg2bFje8dHR0ejfvz80Gg0iIiLg5eWFTZs2ISEhAR999BHGjx9frJ9FeabVapGQkIDg4GBouNcFOSm20wckSVDu3Am3BQugtLRnSQnJ7u4wNm9uklhzGHg+tlNyBWynVNZSUmC2XVfB77OzHf83o3CCXTDJrlTJ6dO0CsslkujIyEisXbv2vscUZzunuLg4BAUFWUygc/3333+oV69eqWN69tlnsWTJEpOyY8eOYdasWThy5Aj0ej3CwsIwduxYREREFHm/ioB/TMkVsJ2WnnjiBNwWLoRq40YIRvOehQeVNwz8kUfyh4H7+trs+q6E7ZRcAdspOZIkAdeuWd66Kz5exJUrAmTZsUl2pUr5SXXhedhMsB3LJZJoqhj4x5RcAdup7QgJCXD76iuo1q2DePOmza9vNgz84YchNW5cIYaBs52SK2A7JWeWnQ0kJuYk1efPG3HqVCZu3fJDXJwKly6J0Okcm2D7+eUk00ywHYNJNDkN/jElV8B2WgZkGWJcHBRHj+Y8jh2D4tQpCDqd7W9VeBj4ww9DDgkpd8PA2U7JFbCdkqso3FaNxpy9sS9dEnHxogKxsSIuXhRx6VLOw9HDxHMTbEtDxP39mfrZgkssLEZEROWYIECqXRtS7drQDxyYU6bTQXH6tGliffFi6W+VlQXl4cNQHj6cVyZVrQrjww+brgZeQYeBExFR0RQKICRERkiIEZ06mU5LMhqBpCThXmLtmAQ7JUXEsWMijh0zr/Pzsz5EnAl28bEnmpwGP5EmV8B26jjCnTtQHD+en1QfPQrx9u0yuZex8GrgLjYMnO2UXAHbKbkKW7VVScrpwY6NFREbq8DFi+K9r52nB/t+c7DL2aCtUmFPNBERuQS5UiUYunaFoWvXewWFhoEfPw7FyZM2GQauOH8eivPngXsLSMoaDYwtWuT3WD/ySLkcBk5ERGVHFIHgYBnBweY92IUT7II92LGx9uvBPn5cxPHj5nW+vjLq1jVaHSJe0f4cMokmIiLXVJxh4MePQ/Hff6W/lVZrPgy8SpW83mrjI4/A0LIluOEnERE9iKIS7KQk4V7PtfkQca227DPY1FQBx48rrSbYdeoYrQ4RL48JNodzk9PgsC5yBWynrqciDgNnOyVXwHZKrsKZ22rBBLvgQme5Q8TtkWDfT3lNsNkTTURE5ZqlYeDC5ctQFl4NPDu71PeyOAy84GrgHAZOREQ2JIpAUJCMoKDcHmx9Xl1ugp2bVBech23PHuy//1bi779NywVBxtWraXCyzySKjUk0ERFVLIIAOTQU+tBQ6AcMyCnLHQZ+r6daceyY7YaB//knlH/+mVcmVamSN7fa+OijHAZORERlomCC3bFj8RLs3DnYZZ1g16wpu2wCDTCJJiIiAtTqnMT24YeBV14BUGgY+L1/xVu3Sn0r8eZNiDt2QLVjR16ZsUEDk222XG01cCIici1FJdhXr1oeIm6rBLtOHanU13AkJtFEREQWOGQY+A8/5Nyq8DDwhx+GXKsWh4ETEVGZE8WcnuKaNa0n2JaGiJckwa5b11j0QU6MSTQREVFxWBsGfuZM/mrgHAZORETlWMEEu0OH4ifYly6JyMrKT7DZE01ERFRRqdUwtmwJY8uWecPAkZICZeHVwMtqGHj9+vm91RwGTkREDlRUgn3tWv4Q8Vat2BNNREREufz8YOjSBYYuXXK+LzgM/NixnMfJk7YZBn7hAhQXLpgOA2/WDMoWLVCpVi0oOnYEwsIAhaLU9yIiInpQogjUqCGjRo3cBNu1MYkmIiIqS0UNA89NrC9cKP2ttFoojxyB8sgR1L1XJms0MDZqBKlxYxibNIGxcWNIjRtDrly51PcjIiKqiJhEExER2Zsdh4ELWi2UJ04AJ06YlEvVq+cl1MZ7D6l+fUCtLvU9iYiIyjMm0URERM7A2jDwgntX22gYOACIV69CvHoV2LUrr0xWqSA1aJCTVDdtmpdgy1WrcmVwIiKie5hEExEROaOCw8D7988p0+kgxsSYbrNlg2HgebfU63OGmZ85A6xfn1cuBQSY9FgbGzeG1KgRoNHY7N5ERESugkk0ERGRq1CrIbVoAV2LFsDIkTllucPAC/RYi8nJNr2tmJwMce9eKPfuzSuTFQpI9erlDwm/N99arlmTvdZERFSuMYkmIiJyZRaGgevOn0dqdDSq3bgBt3PnIJ45AzE2FoIs2+y2gtEIxblzUJw7B/z8c1657Oub32PdpElOgv3QQ4Cnp83uTURE5EhMoomIiMoTQYBUqxZSOneGd3AwjLlDrjMyoDh7FuKZM1CcPp0zbPv0aQipqba9fWoqlAcPQnnwYF6ZLAiQatc2XcSsSRNItWrl7HtCRETkQphEExERVQSenjA+8giMjzwCfW6ZLEO4ciVvHrSY+++FCxCMttvHU5BlKGJjoYiNhWrz5rxy2csLxrAw01XCw8IAX1+b3ZuIiMjWmEQTERFVVIIAOSgIhqAgGLp3zy/XaiGeO2eaXJ8+bfO51kJ6et6+1gVJwcGmw8EbN4ZUty6gUNj0/kRERA+CSTQRERGZ0mggNW8OqXnz/F5rAMKNGznJdIEh4eK5cxD0equXehBiQgLEhASotm/PK5M1GhgfeshsSLjs72/TexMRERWFSTQREREVi1y1as4CZrmLmAGAXg/xwgWT4eCKM2cgJiXZ9N6CVgvl338Df/9tUi5Vr246HLxxY0j16wNqtU3vT0RElItJNBERET04lQpSWBiksDBg4MC8YuH2bYi5C5jlJtj//gtBq7Xp7cWrVyFevQrs2pVXJqtUkBo2NBsSLletyu23iIio1JhEExERkc3J/v4wduwIY8eO+YVGI8TYWNMVws+cgRgfb9N7C3p9zvVPnwbWrcsrlwICTHqsjY0bQ2rUCMhdwZyIiKgYmEQTERGRfSgUkOrXh1S/Pgx9++aXp6ZC8e+/JouYKWJiIKSn2/T2YnIyxL17ody7N69MVigg1atn0mNtbNwYcs2a7LUmIiKLmEQTERGRY/n6wti6NYytW+eXSRKE+HjTHuszZyDGxkKQZZvdWjAaoTh3Dopz54Cffsorl31983uscxPshx4CPD1tdm8iInJNTKKJiIjI+Ygi5NBQGEJDYejdO788IwOKs2fze6zvJdlCaqpNby+kpkJ58CCUBw+alMs+PpCqVIEcEAA5IMDka7lKFUi5XwcEQK5cGVDyrRYRUXnDV3YiIiJyHZ6eMD7yCIyPPJK//ZYsQ7hyxXQRszNnIF64AMFotOnthbQ0KNLSgIsXizxWFgTIlSpZTrILfS0HBECuVAkQRZvGS0REtsckmoiIiFybIEAOCoIhKAiG7t3zy7VaiOfOmSbXp09DTE62T1iyDOH2beD2beD8+SKPlxUKyJUr5/dyF0qypUJfw9eX87YrCkkC0tMhpKVBuHs3/9+7d4G0NJNy6HSQa9bMn98fFMR2QmRjTKKJiIiofNJoIDVvDql5c9Ne6xs3TBcxO3MG4rlzEPT6+12tzAlGI4QbN4AbN4p1vKxSWR5WXjjhvvcvPD2ZTNmbLAMZGUUmvnkJ8L36gsfmfv3AIfj45K9GHxaW8/VDDwE+PjZ8okQVC5NoIiIiqjgEAXJgIAyBgUCXLvnlej3ECxdMhoMrzpyBmJTkuFiLIOj1EK5eBa5ehaIYx8saTdHzuAt8D3f3Mn8OTkuWAa32wRLfAsfh7l0IkuTQpyKkpUF56BCUhw6ZlEshIfnJdePGMIaFQapbl/P4iYqBvyVEREREKhWksDBIYWHAwIF5xcLt2xBze6sTEiAkJ0NIToZ48yaEW7cg3Lzp8B7s4hK0WgiJiRATE4t1vOzlZTXBNhlinpt0q9Vl/AyKKTs7P/FNTTXr0TVLfAslxsj92mBw9DMpU2J8PMT4eKi2bcsrk93cIDVqBOO9Huvcbd/kqlUdGCmR82ESTURERGSF7O8PY8eOMHbsaOUAGUhNhZibUN9LrsXCX99LvoXkZIf3TBaXkJ4ORXo6EBdXrONlX1+zedvW5nTL/v7mF9Drc/YGf5DEt+Dw5+xs2/4gKhAhOxuKkyehOHnSpFyqUiV/KHhYWM62bw0bVuzRClShMYkmIiIielCCAPj5QfLzA+rWLfp4SYKQknL/hLtAL7d4+3aZPwVbEVJToUhNLfbK5d6VKsHXwwMqnQ7i3bsQsrLsEGX5J4uizT+oEW/ehLh3L5R795rcR6pb13SudePGkENCuMo8lXtMoomIiIjsRRQh+/vn9MQ2bFj08QYDhNu3cxLr5GSIycl5X+cNKy/4dVpa2T8HG8hduVzjQh8SlDXZ3R2yjw9kb2/I3t5A7tcFymRf35y6wuX3voa3NyBJOfP7Y2Jy5vfHxORMRyjmMP7iEiQJigsXoLhwAfjll/zn4eWVPxy8QO81/Pxsen8iR2ISTUREROSslErIVavmzUktctfr7Oz8Xuzc4eP3G2KekVHmT6G8k93c8hJZFEpqZR8fkwRXtpAYw9cXspcXoFLZJiCFIn9+/4AB+eUpKfkJ9b1/FTExOUPobUhIT4fyyBEojxwxKZeCgvIS6ryFzOrXt93zJrIjJtFERERE5YWbG+QaNSDXqIFiDejNzMzv4S7Qy23W4537fTmabywrlaVLfHO/dnNz9FMpHj8/GNu2hbFt2/wySYIQH5+XUOeuTC9evGj7IeH3FrVT7diRVyarVJAaNsyfZ32v51quVo3bsZFTYxJNREREVFF5eEAOCYExJKToY2UZSE+/75Bykx7w5OQyWeFaFkXTxLdQkgsLw5xlHx+zcmg0TNREEXJoKAyhoTD06pVfnpUF8dy5vK3ecrd+E5OTbXp7Qa/P2av99Glg/fq8csnf32SetdS4MYyNGuXsdU7kBJhEExEREVHRBAHw9obk7Q3Url308bkrl+cm2YWGmMvXryM7ORlulStDrFTp/olvwV5gT08mv2XN3R1SixaQWrRAwQ3chBs3coaC39v2TRETA/HsWZuPUBBv34a4fz+U+/fnlcmCAKl27byh4MbGjSE1aQIpNJQLmZHdMYkmIiIiItsruHJ5/fpm1VqtFgkJCQgODoZGo7F/fFRictWqMFatCmPnzvmFBgPE2Ni83uq8nuv4eJveW5BlKGJjoYiNhWrz5vyYPDxgfOghs55ri9uoEdkIk2giIiIiInowSiWkBg0gNWgA9OuXX56WBsW//5ouZHbmjM1XkBcyM6E8dgw4dsykXKpe3WyFcKlBA9eZw05OjUk0ERERERHZlo8PjI8/DuPjj+eXyTKExETThcxiYiCePw/BWOTa8yUiXr0K8epVYNeu/NvfS/gLrhBubNwYcs2anCJAJcIkmoiIiIiIyp4gQA4OhiE4GIann84vz87OWcis0BZc4rVrtr29wZBzj5gYYMOGvHLZ19dshXDjQw/l7LtNZAGTaCIiIiIichw3N0jNmkFq1sx0IbNbt/LnWef2XP/7L4SsLJveXkhNhfLQISgPHTIpl2rVgrFxYygaNoS/vz9UDz8MoUEDyAEB7Lmu4JhEExERERGR05ErV4axY0cYO3bMLzQaIcbFQTx9Or/n+swZiHFxEGTZpvcXL1+GePkyVFFR8CoYl5cXpFq1IIWGmj9CQjjvugJgEk1ERERERK5BoYBUty6kunVh6NMnvzw9HYqzZ0224BJjYiDeuWPzEIT09LyF0gqTBQFyzZrmSXbt2pBCQyFXrsxe7HKASTQREREREbk2Ly8YH30UxkcfzR8SLssQrl7NHwqeu/3W+fMQ9Pr7Xe2BCfcWTxMTE4EDB8zqTXqx7yXWeY/gYPZiuwgm0UREREREVP4IAuQaNWCoUQPo1i2/XKeD+N9/+Xtb5w4Lv3Kl7EMqTi+2pWHitWvn7H3NXmynwCSaiIiIiIgqDrUaUlgYpLAwYODA/PKUFNNFzHIXMktPt0tYJr3Y+/eb1cve3laHiUvBwYBabZc4iUk0ERERERER4OcHY7t2MLZrl18mSdDFxuL20aOolpkJt6QkiJcu5SxuFhcH8dYtu4Un3L2bM9/79GmzOlkUIdeoYXmYeGgoe7FtjEk0ERERERGRJaIIKSgId2UZfsHBgEZjWp+WlrOK96VLOf/mJtdxcRDj48ts7nVhgiTdvxfbx8fyiuK1a0MKCmIvdgkxiSYiIiIiInoQPj6QmjaF1LSpeZ3RCOHKlfykOjfJzu3Jvn3bbmEKaWlQ/PMPFP/8Y1Yni6L5XOwCvdlypUrsxS6ESTQREREREZGtKRSQQ0JgDAkx3es6V2qqee91wV5sg8EuYQqSBCEhAWJCArBvn1l9Xi+2hWHiFbUXm0k0ERERERGRvfn6QmrWDFKzZuZ1BgOE3PnXlhJtZ+rFDgqyvqK4n1+57MVmEk1ERERERORMlMr8XmxL9Skp+cl14WHiCQn27cWOj4cYHw9ER5vVyz4+VoeJS7VqAaJolzhtjUk0ERERERGRK/Hzg+TnB6l5c/M6gyF/LnbuomcFe7Hv3LFbmEJaGhSnTkFx6pRJuaxUIu3aNSbRRERERERE5GBKJeRatWCsVct6L3bBHuyCDzv1YkvBwYDSdVNR142ciIiIiIiISsbPD1KLFpBatDCvMxhytsoqnGDn9manpNgkBCk01CbXcRQm0URERERERJTTix0aCmNoKIydOpnXF+zFLjxMPCEBgtFi37cZJtFERERERERU/hWnF9vCll2KS5cgpKbmHcokmoiIiIiIiCq2gr3Ylupze7Hj4iCFhdk7OptiEk1ERERERERl63692C7GNdcUJyIiIiIiInIAJtFERERERERExcQkmoiIiIiIiKiYmEQTERERERERFROTaCIiIiIiIqJiYhJNREREREREVExMoomIiIiIiIiKiUk0ORWFQuHoEIiKxHZKroDtlFwB2ym5CrZVKkhISUmRHR0EERERERERkStgTzQRERERERFRMTGJJiIiIiIiIiomJtFERERERERExcQkmoiIiIiIiKiYmEQTERERERERFROTaCIiIiIiIqJiYhJNDnf8+HEMHDgQISEhqFGjBrp164aNGzc6OiwiAEBSUhIWL16Mfv36oUmTJqhSpQoaNGiAF154AUePHnV0eET3NXfuXPj5+cHPzw9//fWXo8MhMrF582b07dsXtWvXRmBgIJo1a4YRI0YgMTHR0aERQZZlbNq0Cb1790bDhg1RvXp1PProo5g4cSLi4uIcHR45mNLRAVDFFh0djf79+0Oj0SAiIgJeXl7YtGkThg8fjsTERIwfP97RIVIFt3TpUsydOxe1a9fGE088gYCAAFy8eBFbt27F1q1bsXz5ckRERDg6TCIzMTExmDVrFjw9PZGRkeHocIjyyLKM119/HStXrkTt2rXRv39/eHl54erVqzhw4AASEhIQFBTk6DCpgps6dSoWLVqEatWqoVevXvD29sbp06fx7bff4qeffsKOHTsQFhbm6DDJQYSUlBTZ0UFQxWQwGNCqVSskJSXht99+Q7NmzQAAqamp6Nq1K+Lj43H06FGEhIQ4OFKqyDZt2gR/f3+0b9/epPzgwYPo06cPPD09ce7cObi5uTkoQiJzer0e3bp1g0qlQp06dbB+/Xr89ttvaNWqlaNDI8KSJUvwzjvvYOTIkfj000+hUChM6g0GA5RK9vOQ41y/fh0PPfQQatasif3798PX1zevbtGiRXjvvfcwdOhQLFq0yIFRkiNxODc5THR0NC5duoQBAwbkJdAA4Ovri0mTJkGn02Ht2rUOjJAIeOaZZ8wSaABo27YtOnTogJSUFMTExDggMiLrPv/8c5w9exYLFy40S1CIHCkrKwuffvopQkND8cknn1hsn0ygydHi4+MhSRJat25tkkADwNNPPw0ASE5OdkRo5CT4KkUOs3//fgBAly5dzOq6du0KADhw4IBdYyIqCZVKBQBMUsipnDhxAl988QXeffddNGrUyNHhEJnYvXs3UlJSMHToUBiNRkRFReHixYvw9fVF586dUadOHUeHSIS6detCrVbj8OHDSEtLg4+PT17d9u3bAQCdOnVyVHjkBJhEk8NcvHgRQM4LVWGBgYHw8vJCbGysvcMiKpaEhATs2bMH1apVQ+PGjR0dDhEAIDs7G5GRkWjatCkmTJjg6HCIzJw4cQJAzoeP7dq1w3///ZdXJ4oiXn31VcyYMcNB0RHl8Pf3x7Rp0zB16lQ89thj6NmzZ96c6OjoaIwcORKjRo1ydJjkQEyiyWHS0tIAwOTTvYK8vb3zjiFyJnq9HqNHj0Z2djamT5/OnmhyGjNnzsTFixexZ88etktySrlDYBctWoTmzZtj9+7daNCgAU6dOoWJEydi4cKFqF27NkaMGOHgSKmiGzt2LGrUqIHXXnsNX3/9dV55mzZtMGDAAE47qOA4J5qIqAQkScKrr76KgwcP4sUXX8SQIUMcHRIRAODIkSNYsGAB3nzzTa4YS05Lkv6/vXuPqbr+4zj+krvjEFYCNvCCA42jltMowGmzeZkgldCY5Lz8IeGZkYmla3OVxTJa4RZecaNAQRY0RJvWENDTPM20TSNZgEqglMZMQNr0APL7w3HyyKWT9eOgPB8bG/tc+L75C17n8/l8P7ckSR4eHsrLy9O0adNkMBgUFRWlzz//XC4uLtq6dauTqwSk9PR0vfLKK0pNTdXZs2d16dIlHT58WDdu3NDChQt16NAhZ5cIJyJEw2m6V6D7Wm2+fv16n6vUgDPcunVLq1evVmFhoRISErRlyxZnlwRIuv02Y5PJpEmTJmnt2rXOLgfoU/ff9alTp+qxxx6z6zMajRo3bpzq6urU3NzshOqA244eParNmzcrKSlJa9euVWBgoAwGgyIjI1VQUCB3d3dt3LjR2WXCidiHAKfpPgt9/vx5TZ061a7vypUramtr07Rp05xQGdBT9wp0QUGBXnrpJe3YsUMuLnwOicGhra3N9p4JPz+/XsfMnTtXkrR3714tXLhwwGoD7hQaGipJPd543K27/caNGwNWE3C30tJSSdLMmTN79AUEBCg0NFQ//vij2traZDAYBro8DAKEaDjNjBkzlJGRofLycsXHx9v1lZWV2cYAznZngI6Li9OuXbs4b4pBxdPTU0uXLu21z2Kx6Pz581qwYIFGjhypMWPGDHB1wF+6Q0lNTU2Pvvb2dl24cEHe3t4aOXLkQJcG2FitVkl9X2N19epVubi42G7pwNBDiIbTPPvssxo3bpyKioqUnJxsuyu6paVFGRkZ8vDw4LwpnK57C3dBQYFefPFFZWVlEaAx6AwfPlyZmZm99plMJp0/f16pqakKDw8f4MoAe8HBwXruuedUXl6u3NxcLVu2zNa3ZcsWtbS0KCEhgZc2wakiIiK0e/dubd++Xc8//7zdzons7Gw1NjYqIiJCnp6eTqwSzjSsubm5y9lFYOgym82Kj4+Xl5eX4uLiZDAYdODAAV28eFHvv/++UlJSnF0ihrjNmzcrPT1dBoNBq1at6jVAx8TE2D4EAgYbk8mkffv2qbS0lBCNQaGurk7z5s1TU1OT5s+fb9saazabNXr0aB05ckQBAQHOLhNDWGdnp2JjY2WxWOTn56cFCxbI19dXZ86ckdls1vDhw/XVV19p+vTpzi4VTsLHfHCqWbNm6euvv9bmzZtVXFys9vZ2GY1Gbdq0SXFxcc4uD1BDQ4Ok22dOP/74417HjBkzhhANAA4KDg5WRUWFPvjgA5WVlam8vFwBAQFKSkrS+vXr+zzXDwwUV1dXFRcXa/v27SouLlZRUZGsVqv8/f2VkJCgdevWaeLEic4uE07ESjQAAAAAAA7i1bIAAAAAADiIEA0AAAAAgIMI0QAAAAAAOIgQDQAAAACAgwjRAAAAAAA4iBANAAAAAICDCNEAAAAAADiIEA0AAAAAgIMI0QAAAAAAOIgQDQAA7lv19fUaMWKERowY4exSAABDhJuzCwAAAP9fMTExOn78uENjm5ub/7/FAABwnyNEAwAwRAQFBSkoKMjZZQAAcF8jRAMAMEQsWbJEb731lrPLAADgvsaZaAAAAAAAHESIBgAAPdz9wq7Dhw8rJiZGY8eOVWBgoObMmaMvvvii359RXV2t1atXa8qUKfL399fYsWMVHR2t3NxcdXZ29jmvo6ND+fn5iouLU0hIiPz9/RUWFqbo6GhlZmaqpaWlz7nfffedEhISFBwcrFGjRikqKkpZWVnq6urqdfzp06eVlJSkyZMny9/fX4GBgZoyZYri4+OVmZnZ5zwAwNDFdm4AANCvXbt2acOGDXr44Yc1fvx4NTY26tSpU7avjz76qMec4uJiJScny2q1ytvbW0ajUdeuXZPFYpHFYlFxcbHy8/M1fPhwu3lXr17Vyy+/rBMnTkiSAgICNHnyZDU1NenEiROyWCyaOnWqZs6c2eOZeXl5SklJka+vr8aNG6eLFy+qqqpK69evV0NDg9LS0uzGHzlyRImJiWpvb5fBYFBISIjc3Nz066+/qqysTGVlZTKZTHJz498lAMBfWIkGAAD92rhxo958803V1taqoqJC1dXVysjIkIuLi7KysrR//3678TU1NTKZTLJarVq2bJlqamp09OhRnTlzRvv379dDDz2kiooKvf3223bzurq6tHz5cp04cUJBQUEqKSlRdXW1ysvLVVlZqQsXLuiTTz6Rv79/r3WmpqYqLS1N586dU0VFhc6dO2d7xrZt21RXV2c3/t1331V7e7vWrFmj2tpaWSwWmc1mnTt3TpWVldq0aZNcXPhXCQBgb1hzczP7lAAAeIA5esVVdHS08vPzJd3ezv3kk09KkmbPnq3i4uIe49esWaOcnBwZjUZZLBZb+6uvvqq9e/fKaDTq+PHjGjZsmN283Nxcvfbaa3J3d1dlZaVGjRol6faW8cTERHl6espsNmvixIl/W/OddS5dulSZmZk9xkRFRamqqkoffvihVq1aZWsPCAjQzZs3VV9fL19f3799FgAAEivRAAAMGUFBQYqIiOjz6/HHH+91nslk6re9qqpKly5dsrWXlpZKklatWtUjQEtSYmKi/Pz81N7eroqKClv7gQMHJEmxsbEOBei7rVy5stf2p59+WpJ04cIFu/bRo0dLkoqKiv7xswAAQxeHfAAAGCLu9YqrsLCwXttDQ0Pl5uamjo4O1dTUKCgoSC0tLbpy5YokyWg09jrP3d1doaGhampqUm1tra29qqpK0l+h958KCQnptd3Pz0+S1NbWZte+Zs0apaSkaN26ddq6datmz56t8PBwzZgxQ2PGjLmnGgAADz5WogEAQL/6OoPs6uqqRx55RJJ0/fp1SfZBta95kmxbuLvn3fn9vW6t9vb27rW9+1zz3W/aXrp0qfbs2aNnnnlG9fX1ys7Olslk0hNPPKE5c+bo22+/vac6AAAPNkI0AADo1++//95re2dnp/744w9Jko+PjyTJYDD87TxJunz5st28O7/v7wqr/1psbKy++eYb1dXVqbCwUGvXrlVwcLBOnTql+Ph4VVZWDlgtAID7AyEaAAD06+eff+61vba2Vh0dHZKkCRMmSLq9ihwQECDpr+3Zd+vo6LBt4+6eJ0mTJk2SJH3//ff/TeH/gK+vr+bOnat33nlHJ0+eVHh4uKxWq3Jzcwe8FgDA4EaIBgAA/dq5c2e/7UajUUFBQbb2efPm2frv3kItSQUFBWpqapK7u7tmz55ta3/hhRckSQcPHrQ7Kz3Q3NzcNH36dEnSb7/95rQ6AACDEyEaAAD0y2w2Kz093bbq3NXVpZycHO3Zs0eS9MYbb9iNT0lJkZeXl6qqqvT666/rzz//tPUdO3ZMGzdulCStWLHCtmotSfPnz9esWbN08+ZNxcXF9TiT3NraquzsbFVXV//r36m1tVXLly9XWVmZrFarXd/p06dtV3pNmzbtXz8LAPBg4e3cAAAMEXl5eTp27Fi/Y9LT0233LndLS0vThg0btHPnTgUHB6uxsdH2Bu6VK1cqLi7ObvyECRO0Y8cOJScnKycnR19++aVCQ0N17do1/fLLL5Ju3z393nvv9Xj+Z599psWLF+vkyZOKjY3VqFGjFBgYqKamJjU2Nqqzs1MHDx68pyuw7nTr1i2VlJSopKREHh4eGj9+vLy9vdXU1KSGhgZJ0lNPPWV3rzQAABIhGgCAIePSpUt29zn3prW1tUdbcnKyRo8erW3btqmyslIdHR2aPn26kpKStHjx4l5/zqJFixQWFqZPP/1UZrNZZ8+elZeXlyIjI5WYmKglS5bI1dW1x7xHH31Uhw4d0r59+1RYWKiffvpJlZWVGjlypCIiIhQTE9Mj5N8LHx8f7d69W2azWT/88IMuX76slpYW+fj4KDIyUosWLdKKFSvk4eHxr58FAHiwDGtubu55WAkAAAxp9fX1trDa3Nzs3GIAABhEOBMNAAAAAICDCNEAAAAAADiIEA0AAAAAgIMI0QAAAAAAOIgXiwEAAAAA4CBWogEAAAAAcBAhGgAAAAAABxGiAQAAAABwECEaAAAAAAAHEaIBAAAAAHAQIRoAAAAAAAcRogEAAAAAcBAhGgAAAAAABxGiAQAAAABw0P8A1ZF0rNtgt8kAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1000x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = sbs_transf.plot_losses()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "HPQvdYWVPIpY",
    "outputId": "f12ce16d-b672-424f-dfa6-51f97417686e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[437, 453],\n",
       "        [288, 318]])"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "StepByStep.loader_apply(test_loader, sbs_transf.correct)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8Xh8xxgTPIpY"
   },
   "source": [
    "### Visualizing Attention"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "i5_67okaPIpY",
    "outputId": "6ecffbd1-7e4e-490f-f239-0fd1117048bd"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[    2,   300, 12427,     7,  7101,  1423,   422],\n",
       "        [    2,  6659,   811, 11238,    15,     2,   588]], device='cuda:0')"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentences = ['The white rabbit and Alice ran away', 'The lion met Dorothy on the road']\n",
    "inputs = glove_tokenizer(sentences, add_special_tokens=False, return_tensors='pt')['input_ids']\n",
    "inputs = inputs.to(sbs_transf.device)\n",
    "inputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Cu3MXRxgPIpZ",
    "outputId": "6d2a66c7-5881-4b14-9dca-cbc99bfb1197"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.9893],\n",
       "        [0.0094]], device='cuda:0', grad_fn=<SigmoidBackward0>)"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sbs_transf.model.eval()\n",
    "out = sbs_transf.model(inputs)\n",
    "# our model outputs logits, so we turn them into probs\n",
    "torch.sigmoid(out)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4Nld9xIhPIpZ",
    "outputId": "0c12f02c-d6df-4326-d423-45ca7a13c711"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[2.7641e-01, 5.6965e-02, 1.7406e-01, 1.6204e-01, 8.9550e-02,\n",
       "          1.5953e-01, 6.5942e-02, 1.5511e-02],\n",
       "         [3.3255e-05, 3.0550e-03, 4.6508e-03, 7.8658e-02, 2.6481e-03,\n",
       "          9.0915e-01, 7.1525e-04, 1.0912e-03]],\n",
       "\n",
       "        [[7.4745e-02, 1.5404e-02, 1.0670e-01, 5.9543e-02, 6.3095e-01,\n",
       "          9.5393e-03, 1.3835e-02, 8.9286e-02],\n",
       "         [3.3745e-04, 3.1001e-02, 2.8195e-01, 5.8777e-02, 4.9707e-01,\n",
       "          6.2487e-02, 3.8411e-02, 2.9971e-02]]], device='cuda:0')"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "alphas = sbs_transf.model.encoder.layers[0].self_attn_heads.alphas\n",
    "alphas[:, :, 0, :].squeeze()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 259
    },
    "id": "B5gATXCkPIpZ",
    "outputId": "a7bee94b-9934-4d21-f5f7-db27f5ab2c4f"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/content/plots/chapter11.py:50: UserWarning: FixedFormatter should only be used together with FixedLocator\n",
      "  ax.set_yticklabels(['', f'Head #{i}', ''])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABbIAAADNCAYAAABtqYY+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACFhElEQVR4nO3ddVgU2/8H8PeSiqCgAiZtXAsDE7HAAlSwu64d185r57U7sO61A8HEDhAQVOxOwC4UKQGJ3x/8Zr6su8ACCyz6fj2Pz6Ozs2fPHGfOnPnMmc9IwsPDk0FEREREREREREREpKLU8roCRERERERERERERETpYSCbiIiIiIiIiIiIiFQaA9lEREREREREREREpNIYyCYiIiIiIiIiIiIilcZANhERERERERERERGpNAayiYiIiIiIiIiIiEilMZBNRERERERERERERCqNgWwiIiIiIiIiIiIiUmkMZBMRERERERERERGRSmMg+xewe/du6Ovri3/69++f11VKl4ODg1R9fX1987pKChs6dCj09fURGhqa11VROb6+vtDX18fChQsV/k7VqlVRtWrVHKyVasqNthL6hd27d2elir80fX19ODk55XU1siWtfcjJyQn6+vp5UykiIlIKju1zD8f2QGhoKPT19TF06FBxGduF5O0XqmzhwoX5rv+h7GE/pTpy+/hjIPsX4ujoiEmTJqFdu3Yyn8XExGDDhg1wdnaGpaUlihcvDjMzM7Rq1QorVqzA58+fpdbX19dH7dq1Ffrd6OhoLFu2DI0aNULp0qVhZGSESpUqoXXr1pg9ezaCg4Ol1u/duzcmTZoEW1vbrG9sDspKkJGUh4OQnPUrBHGJiIh+BxzbKwfH9pQfcVKKtPwWWCflYP9N8mjkdQVIeZycnNCjRw+Z5Xfv3kX37t3x6tUrlC1bFq1bt4aRkREiIyNx7do1zJ49GytWrMDDhw9RqFChTP1mZGQkWrVqhfv378PCwgKdO3dG0aJFERYWhuvXr2PFihUwNzeHubm5+J3evXsDSAlY+vv7Z2+jKV87evRoXlfhl+Xs7IzatWvD2Ng4r6tCuWjjxo34/v17XleDiIiUgGN7yiszZ87EmDFjUKpUqbyuChERkRQGsn9xb968Qfv27REWFoZ58+Zh6NChUFdXl1rn9u3bmDhxIn78+JHp8jds2ID79++jd+/eWLVqFSQSidTnISEhiI+Pz9Y20K8r9UUQKVeRIkVQpEiRvK4G5bKyZcvmdRWIiCgHcWxPuaFEiRIoUaJEXleDiIhIBlOL/OLmzp2LT58+YezYsRgxYoTMQBcArK2t4eXlhcKFC2e6/GvXrgEABgwYIDPQBQAzMzOUL18+8xXPIwsXLkSbNm0AAIsWLZLK95c691JycjI2btyI2rVrw8jICFWqVME///yDpKQkueV6eXmhbdu2MDU1hbGxMerXr481a9YgMTExV7ZLEeHh4ShatCi6dOkitfzOnTtiG7x48ULqMycnJ5QoUQJxcXFSy2/evAkXFxeUKVMGJiYm6NGjh9zcVT/nfXZycsKiRYsAAG3atBF/9+fc0J8+fcKUKVNQo0YNGBkZwcLCAr169cKDBw+y1QbpSf1Y05UrV+Dq6goTExMxH/HOnTvRrVs3VK1aFcbGxjAzM0P79u1x6dKldMsNCAiAk5OT2Fa9evWSaefUwsPDMXr0aJQvXx7Gxsaws7PDwYMHZdb7+XFEof4A4O/vL7Vv58Uji/Hx8XBzc0P79u1RuXJlGBkZwcrKCj179sTt27el1k29LRcuXECLFi1QsmRJmJubY8iQIfjy5Yvc39ixYwfq168PY2NjVK5cGTNmzEBsbGxubF6WZKZN0pJejmwvLy+4urrC3NwcxsbGqFq1KgYNGiRz3MTHx2Pt2rVo1KgRSpUqhTJlyqB169Y4ceJEdjcxxynzOE1dlqJ9Wl5SdP+5e/cu9PX1MWHCBKnvHz9+HPr6+jAyMkJMTIzUZ1WrVkW1atXEf3/79g0rV66Eo6MjKlasCENDQ1SsWBGDBw+WSTkwb9486Ovr49ChQ3LrvXPnTujr62P58uXZbQKi3wLH9pnzO4/tsyO93LO7du2Cvb09SpcujdKlS8Pe3l7uWDK/nUfT8vPYwtnZGWXKlIGlpSXGjRsnPgl3+vRpNG/eHKVKlUK5cuUwY8YMJCQkyJSnyL4zdOhQDB8+HAAwfPhwqf1W2RITE7Fy5UrUqFEDxsbGqFGjBpYvX47k5GS56z948AB9+/aFlZUVjIyMUK1aNUyePFnueFy41gsPD8eECRNQuXJlFCtWTGp/OXnyJJydnWFiYoISJUrA1tYWa9eulWq73bt3w9raGgCwd+/eDHPyu7u7o2HDhihRogQqVKiASZMmST2x6O3tDX19fYwbN07uNgYHB8PAwADt27dXrBGVSJHj61c5tjLC/ltWRtc60dHRWLBggfhktpmZGTp37ozAwECZst69e4cFCxbAwcFBPJ6rVq2KcePG4dOnT3J///Xr1/jzzz9hZmaG0qVLw9HRMU+exOKM7F9YTEwMPD09UbBgQYwcOTLddTU0srYrGBgYAACeP38udZGbXzVs2BAvX77E3r17YWtri4YNG4qfpZ7dOmPGDPj7+6Nly5Zo1qwZvLy88M8//+DHjx+YPn26VJnC452lSpVCmzZtULhwYQQEBGD69OkICgrC9u3bc2370qOvr48qVaogICAAiYmJ4oVR6sGBr68vLCwsAACxsbEICgpCnTp1oK2tLa5z8+ZNrF69GnZ2dujbty/u3LkDLy8vPHjwAAEBAShQoECadejevTuAlEBrt27dYGJiAkC67YODg+Hs7Iw3b96gWbNmcHJywqdPn3Ds2DFcuHABR44cgY2NjfIa5idXr17F8uXLxe17/fo1AGDChAmoUqUKmjRpguLFi+Pt27c4ceIEXFxcsHPnTrl5qYOCgrBixQrY29tj0KBBePjwIY4fP46AgACcO3cOZmZmUuv/+PEDLi4uiI6ORpcuXRATE4NDhw5hwIABCAsLw+DBg9Ost4mJCSZNmoRFixahbNmyYlsDyJMXbn79+hVTpkxB/fr10bx5c+jr6yMkJAQnT57EuXPncOLECdSsWVPqOydPnsSZM2fQqlUr1KlTB5cvX8a+ffsQEhKCU6dOSa27ePFiLFiwAEZGRujduzc0NTXh6emJx48f5+ZmZkpW2kRRf//9N9atWwcDAwM4OTnB0NAQb968gY+PD6pXr45KlSoBAOLi4tChQwf4+fmhatWq6NmzJxISEnDmzBl0794dixcvxqBBg5S52TlCmcdpdvq03KTo/lOlShUULVpU5sJP+Hd8fDyuXLmCpk2bAkiZffnq1Sup9AZPnjzBggULYGdnB2dnZ+jo6ODJkyc4ePAgzpw5Ax8fH7H/7t27N5YvX44dO3bA1dVVpt47duyAhoaG3PQJRCSNY/vM+53H9jlh4sSJ2LRpE0qVKoWePXsCAI4dO4bhw4fjzp074oSU1PLLeTQj169fx6pVq9CsWTP07dsXvr6+2Lp1q5iOZ9iwYXB0dETt2rVx5swZrF69GoUKFcKkSZPEMhTdd5ycnPDt2zecOHECjo6OOTpWHzVqFHbt2gVTU1MMGDAAcXFxWLduHa5cuSKzbkBAADp06ID4+Hi0a9cOJiYmuHr1KjZu3IjTp0/j3LlzKFasmNR34uPj0bZtW0RHR6N169ZQV1eHkZERAGDt2rWYNm0aDAwM0LFjR+jo6ODkyZOYNm0aAgICsGvXLkgkElStWhVDhgzBxo0bUaVKFanxmjDeEGzevBnnz5+Ho6MjGjVqhPPnz8PNzQ1fvnzB5s2bAQCNGzeGubk53N3dMXfuXOjo6EiVsWPHDiQnJ6NPnz5KaWNFZfb4+lWOrbSw/06bvGud2NhYtG3bFtevX4e1tTWGDh2Kjx8/4tChQzh//jy2bt0KFxcXsYzLly9j3bp1aNSoEWrVqgVNTU3cuXMHW7duxfnz5+Hj4yPVzu/fv0eLFi3w9u1b2Nvbw9raGo8fP4arqyvs7OxydfsZyP6F3bhxA/Hx8ahfv36OpRhwcXHBgQMH8Ndff+H69eto1qwZqlevjqJFi+bI7+U04QDcu3cvGjZsiClTpshd7/bt2/D39xcfuZs4cSJq1qyJTZs2YdKkSdDS0gIAXLx4UQxU7tixQ8xTmJycjHHjxmHbtm04cuSI3Jf45AU7OzvcuXMHt27dQq1atQCkBDesrKwQGxsLX19f8YR+5coVxMXFyXRaZ86cwbZt26TuYA8ePBj79++Hl5cXOnTokObv9+jRAy9fvoS/vz+6d+8ut0McMmQI3r9/Dw8PD9jb24vLJ0yYgKZNm+Kvv/7C5cuXs9UO6bl48SLWrl0rDi4EgYGBMoHn9+/fo2nTppgxY4bcANn58+exYsUK9OvXT1z277//YsyYMZg0aRL2798vU56FhQXOnDkj7mNjx45Fo0aNMGPGDLRp0ybNXIampqaYMmUKFi1aBBMTkzT37dyir6+Pe/fuydT34cOHaN68OebMmYPDhw9LfXbq1CkcP34c9erVA5Ayg6Rdu3bw8/PDtWvXxJdYvXjxAosXL0apUqXg4+MDQ0NDAMDkyZOl9hlVk5U2UcSpU6ewbt06VKpUCcePH5fqnxMSEqRm0CxevBh+fn6YMGECpk6dKs7Gi4yMRNu2bTFt2jS0adMGJUuWzNpG5hJlHqfZ6dNyk6L7j0Qiga2tLY4dO4aPHz+KF5O+vr6oX78+rl+/Dl9fXzGQLQS4U/fH5cuXx+PHj8WAl+DSpUtwcXHB0qVLsXr1agApF5j29vY4d+4cQkNDYWpqKlW3a9euwcnJifn8iRTAsX3m/e5je2Xy9/fHpk2bUKFCBZw5c0bcB6dMmQIHBwe4ubmhXbt2aNCggdT38st5NCPnzp3D7t27xbHCjx8/0KRJE7i7u+P8+fNSEw6mTJmCmjVrYuPGjRg7diw0NTUzte84OzuLgey0cuUrg6+vL3bt2oUqVarg9OnTYp3Gjh0rcx2WlJSEYcOGISYmRuY6bMaMGVi9ejVmzpyJtWvXSn3vw4cPYvkFCxYUlwcHB2PWrFkwNDTExYsXUaZMGQDA9OnT4eLiAi8vL+zfvx9du3ZFtWrVUKRIEWzcuBFVq1ZN9zrG29sb3t7eKFeuHADg+/fvsLOzg4eHB+bMmYOSJUtCIpGgb9++mDlzJg4fPiw1wSchIQF79+6FoaEhHB0ds9iymZeV4+tXObbSwv47bfKudRYtWoTr16+jc+fOcHNzE6/jBg8ejObNm2PUqFGwt7eHnp4eAKBRo0Z4/PgxdHV1pcreu3cvhg4dis2bN2P8+PHi8tmzZ+Pt27eYNm2a1PL//vsPo0ePzsGtlcXUIr+wjx8/AkCOvqTD0dER8+bNQ3JyMtauXYv27dvDwsICNWrUwIQJE/D8+fMc++28NGHCBKm8ccWKFYOjoyMiIyPx9OlTcfmmTZsAACtXrpR62Y5EIsHMmTMhkUjg4eGRexXPgHCyEB6zT0xMxOXLl2FnZ4eGDRvKzM4GIHVnFAAaNGgg8xiW0MHeuHEjW/W7ffs2rly5gm7duskEJK2srNC7d288ePAgR1OMWFtbywTHAMgEx4CU/IJt2rTB8+fP8fLlS5nPraysZO709+nTB5aWljhz5gw+f/4s850ZM2aIJ2MAKF26NIYMGYK4uDiV2pcyoq2tLbdv+uOPP9CwYUNcvnxZJrdnx44dxSA2AKirq6Nbt24ApPctd3d3JCQkYNiwYWIQGwAKFy4sddJVNVlpE0Vs3boVAPDPP//IBCI0NDTEQGZSUhK2bt0Kc3NzqSA2AOjp6WHixImIj4/HsWPHMl2H3KbM4zQn+zRlysz+I/T1Qj8eFhaGhw8fokWLFrCxsZFKtSKvry9SpIhMEBtIGRBXrFgR3t7eUsv79euH5ORk7Ny5U2r5jh07ACDXZzwR5Vcc2+ecX3Vsr0x79+4FkDIxIPWNFH19fXHW8Z49e2S+l1/Ooxmxs7OTuuGtqamJdu3aITk5Ga1atZJ6ak5PTw8tW7bE169f8ebNGwCque/s27cPQErgL3WdSpUqhSFDhkitGxgYiODgYDRv3lzmOmzixIkwMDDAwYMH5ebQnz17tlQQG/jfeH3EiBFiEBtIGc/MmjULgPz9KSNDhgwRg9gAULBgQXTo0AFJSUm4deuWuLxHjx7Q0tKSGZucPn0a79+/R7du3aCpqZnp38+qrBxfv8qxlV2/Y/8t71pn79690NTUFLcl9brdunXDt2/f4OXlJS43NDSUCWIDQNeuXVG4cGGp8Xx8fDwOHToEQ0NDjBgxQmr93r17w9LSUklbphjOyKZsGzFiBPr06YPz58/jypUruHXrFoKCgrB582bs3LkT27Zty9W7mbmhevXqMstKly4NICV3qCAoKAiFChXCrl275JZTsGBBqc41rzVo0ADq6urw9fXFmDFjcOfOHURERKBRo0aIiYnBvn378PjxY1SoUAF+fn4oWLCgTBoPRdsmK4KCggCk5MheuHChzOdCWz59+lRMlaBsaaV2CAkJwfLly3Hp0iW8e/dOJm/4+/fvZR59q1u3LtTUpO8nqqmpoW7dunj+/Dnu3buHJk2aiJ9paGigTp06Mr9dv359ACm5b/OTO3fuYPXq1QgMDMSHDx9kgrRhYWFSgxJF96179+4BgMyMIOB/baWqMtsmirh+/Tq0tbVlbjr97OnTpwgPD0fJkiXxzz//yHweFhYmrqfqlHmc5mSfpmyK7j+pA9lCKpnk5GQ0atQIsbGxWLZsGSIjI6Gnpwc/Pz+Ym5tLXWQK392wYQOuX7+OsLAwqVyWqW+2AUDLli1RqlQp7NmzB1OmTIG6ujri4+Oxf/9+lClTBg4ODjnUIkSUFRzbp/gVxvbKdOfOHQCyk1iA/90glTcWzU/n0fTIS+8hjMnS++z9+/cwMzNTyX0nM2Pm9P7/dXV1UaNGDVy4cAFPnz5F5cqVxc8KFCgg9W9FyqtTpw4KFCiQpWsbRfe34sWLo02bNvDw8MCTJ0/E3P/CTfbevXtn+rezIyvH169ybGXX79h//3ytExERgZCQEFSoUEHc9tTs7Oywfft23L17F127dhWXHz16FP/99x9u376N8PBwqTzh79+/F//+9OlTxMbGolGjRjIpa1LHL3ILA9m/MGGW3du3b3P8t/T09ODi4iLm3Pn27Rvmzp2LLVu2YOTIkXBwcJC5sM3PhMcxUhNySqc++L9+/YqEhAS5+eIE0dHRyq9gFhUuXBjW1ta4cuUKfvz4AV9fX0gkEtjZ2Ykv//L19UXZsmVx/fp12Nrayvy/Kto2WfH161cAKXfKT58+neZ6OdmmqWf4Cl68eIFmzZohMjISdnZ2aNWqFfT09KCmpgY/Pz/4+/vLBMyA/x2jaS3/eQBSrFgxmcB3euursitXrqBt27YAgKZNm6Jdu3YoVKgQJBIJvLy8cO/ePZk2U3TfioiIAJAyQP1ZWm2uCrLSJoqIiIhAyZIl5e47qQnH18OHD/Hw4cM011OlPistyjxOc7JPU6bM7D9//PEHDA0NxdnWvr6+KFy4MKpXr47v379j0aJFCAgIgKWlJd6+fStzMXf48GH069cPurq6aNasGUxMTFCwYEFIJBLs2bMHr169klpfXV0dvXr1wqJFi3D27Fm0atUKx48fx5cvXzBw4MAM900iSsGxfc75Vcf2yhQZGQk1NbU0x1cSiQSRkZEyn+WX82hG0tuO9D4Tbiqr4r4TEREBNTU1mbzWgOyYWfi/lTfGAiCmCPt5HyhevLjcF8emV55EIoGhoSHevXunwFZIy8z+1rdvX3h4eGDHjh2YN28e3r17h3PnzsHW1hZWVlaZ/u3syMrx9ascW9n1O/bfPx83WTk+16xZg+nTp6N48eJo1qwZSpUqJQapN2zYIHVdlN71NZD719gMZP/CatasCS0tLdy6dQsRERFZenN5VhUpUgRLlizB6dOn8erVKzx48EDunbJfnZ6eHiQSCV68eJHXVVGYnZ0dbty4gevXr8PPzw9//PGH2GGZmprC19cXlpaW+PHjR64n9RdOUnn5wjl5A7H169cjPDwcbm5u6NKli9RnY8aMSfNNvsIjwmkt/zn/ZVhYGJKSkmSCPmmtr8qWLVuGuLg4nDx5UmbGR1BQkDhDJCuEvu7z588ys2vTanNVkFNtUqRIEXz8+FHuvpOacHy1bdtWnI2SXynzOM0vMrv/NGzYEIcOHcLbt2/h5+eH+vXrQ11dHbVr10bBggXh6+srXkD+3Nf/888/KFCgALy9vWUeJfT09JRbv969e2Pp0qXYvn07WrVqhR07dkBNTU1uChgiko9j+7yXH8f2yqKnp4ekpCR8/vxZJljy6dMnJCcnyw0oUQpV3HcKFy6MpKQkhIWFyQSofh4zC/+3nz59kluWsP7P+4C8MdnP5f08Xk9OTsanT59yfH+ys7ND+fLlsW/fPsyYMQO7d+9GYmJinqQ84/GV81TxGMyqn4+rzB6fCQkJWLJkCUqUKAFfX1+pfS45OVl8140g9fV1euXnFk6B+YXp6Oigffv2+P79u8xLF36WkJCApKQkpf6+RCKRyj2UXyjzLqaNjQ2+fPmSr/IJCgGLCxcuICAgQCqA0ahRI/j5+Yn5UzNKVZAVQvvL2x+FNCbXrl1T+u9mR3BwMADIPGabnJws943fgitXrshsZ1JSEq5evQqJRIIqVapIfZaQkICrV6/KlBMQEABA/mONP1NTU1P6sZ4VwcHBMDAwkAm4xcTE4Pbt29kqW2g3eS/9FNpKFeVUm9SqVQtxcXHw8/NLd70KFSqgcOHCuHnzZpZycau6rB6n+UVm9x+hb/fw8MCjR4/QqFEjACm5KevUqYNLly6l+S6E4OBglC9fXiaI/f79e4SEhMitX+nSpdGiRQucPXsWV65cgY+PD+zt7VG2bNksbS/R74hj+6z53cf2ylKtWjUAkDueEJYpMhb9XWV238mNmbWZGTOn9/8fHR2NmzdvomDBglL5qdOTXnlBQUGIjY2V2p9yqj369OmDz58/w8vLC7t27YK+vr74hFtu4vElH/tvxRQuXBhmZmZ48eKF3Ke2ft6HwsLCEBERgdq1a8vcOLl58ya+f/8utczKygoFChTAzZs3ERsbK/WZEL/ITQxk/+KERwWWLVuGjRs3yh3Q3rt3D87OzuLjApnx77//pvkigePHj+Px48coUqQI/vjjj0yXnVeEF1gJL+bIjsGDBwNIyTX45csXmc8/fPiAx48fZ/t3lKlevXrQ0NDAtm3bEBkZKQY3gJTAR1hYGHbu3IlChQqlmYc2O4T2f/36tcxntWrVgo2NDQ4ePCh31l9SUlKGwbqcIARiAgMDpZavWLEi3RdPPnv2DNu3b5datn37djx79gwtWrSQ++jOnDlzpF6i8ubNG2zcuBHa2toKvZnawMBAKft2dpUtWxbh4eFSKSwSExMxffr0NO/0KqpTp05QV1fH+vXrpe5KR0REYOnSpdkqOyflVJsMGDAAQMrLY4T0IYKEhATxDrqGhgb69++PV69eYdq0aXKD2Q8ePEjzTr+qy+pxml9kdv8RAtmrVq0CAJm+/u7du7hw4QKsrKxQsmRJmd8KDg6Wmn0RGxuLsWPHpnsTpF+/fkhISEDfvn2RnJyc6/kniX4FHNtn3u8+tlcW4QXbixYtktq3vn37Jj6qL6xDsjK77yhzv02L8ITa4sWLpVIqvH37Fhs3bpRat169ejA3N8fZs2dlXuq8dOlSfPnyBR06dFA45VCnTp2goaGBdevWSaUQiY+Px8yZMwEA3bt3F5fr6+tDIpEovT26d++OAgUKYOrUqQgJCUGXLl1kcgDnBh5f8rH/Vly3bt3w48cPzJ49G8nJyeLye/fuYc+ePShcuLD4wlpDQ0MULFgQt2/fFlPIAkB4eDgmTpwoU7a2tjZcXFzw6dMnmRvpO3bswLNnz3Joq+RjapFfXOnSpeHp6YkePXpg8uTJWL9+PRo3bgwjIyNERETgxo0buHHjBvT09GTeyvvhwwcMHTpUbrnFihXDvHnzcPbsWYwZMwYWFhaoW7cuSpYsiejoaNy5cwcBAQFQU1PDsmXLoK2tnRubqxTly5dHyZIl4enpCW1tbZQqVQoSiSRLqSwcHBwwYcIELFmyBDVq1ICDgwPKli2LL1++4MWLFwgICMC0adNQoUKFHNiSrNHV1UXNmjVx9epVqKmpwdbWVvxMCHx8/vwZ9vb2OfImZzs7O0gkEsydOxePHj1C4cKFUaRIEbH9t2zZgjZt2qB///7YsGEDrK2tUaBAAbx+/RrXrl3D58+f8eHDB6XXKz39+vXD7t270bt3b7i4uKBo0aIICgrC7du30bJlyzTzedvb22PSpEk4c+YM/vjjDzx8+BCnTp1CsWLF5ObuKlGiBGJiYmBra4tWrVohJiYGhw4dwpcvX7Bo0SKUKlUqw7o2atQIhw4dQvfu3VGtWjWoq6ujdevWMrO/c9qgQYNw4cIFtGrVCq6urtDW1oafnx/evXuHhg0bZuuGhIWFBSZOnIiFCxfC1tYWLi4u0NDQwNGjR1G5cmWVfYlHTrVJixYtMHLkSKxZswY1a9aEs7MzDA0N8fbtW1y6dAkjRozAsGHDAABTpkzB7du34ebmhjNnzqBBgwbiug8ePMC9e/dw9uzZNPOvqbKsHqf5RWb3n3LlyqFEiRJ4//49ihYtKtUH2NnZISkpCV++fEG7du3k/tbEiRPRqFEjtG3bFomJibh48SKSk5NRpUqVNNPgCOfAV69ewdjYGK1bt1ZuIxD9Bji2z7zffWyvLLa2thg0aBA2bdqEBg0aoE2bNkhOTsaxY8fw5s0bDB48WOq6gaRldt+pU6cOChYsiA0bNiA8PFyc4DJhwgSl1alRo0bo0aMHdu/ejQYNGsDZ2Rnx8fHw9PSEjY2N1NhITU0N69evR4cOHdCpUye4uLigbNmyuHr1qvhi6FmzZin828L606ZNg62tLVxdXaGjo4NTp07h6dOncHR0lEoFJ1yjXr58GYMGDYKlpSXU1NTQpUsXmdQkmWFgYIB27dph//79AJAnaUUAHl9pYf+tuFGjRuHMmTPYv38/njx5gsaNG+PTp084dOgQEhIS4ObmJqYWUVNTw59//om1a9eiYcOGaNWqFSIjI3Hu3DmULVtWZhILAMyaNQuXLl3CvHnzEBgYiGrVquHx48c4e/YsmjVrhgsXLuTatjKQ/RuoVq0arly5gu3bt+P48eM4ceIEvn37hkKFCqFChQqYNm0a+vXrJ/OoYEREBPbu3Su3zLJly2LevHmYPXs26tWrh4sXL+Ly5ctiALFkyZLo1q0bBg8enO/y56mrq2Pnzp2YOXMmPDw8xIT4nTt3zlJ5f//9N2xtbbFx40b4+Pjg27dvKFq0KExNTTF58mR06tRJmdVXCjs7O1y9ehXVqlWDvr6+uLxkyZKwsrLCs2fPciStCABUrFgR69atw9q1a7Fp0ybExcWhbNmy4snKzMwMvr6+WLt2LU6cOIHdu3dDXV0dxsbGaNCgQZ48CmZtbQ1PT0/Mnz8fx48fF9/ce+rUKZw8eTLNAJmNjQ3Gjx+PefPmwc3NDerq6nBycsKcOXNgZmYms76mpiYOHz6MWbNmYf/+/fj27RvKlSuHxYsXo2PHjgrV9Z9//gEAXLp0CadOnUJSUhJKlSqV64HsVq1aYfv27Vi+fDkOHDiAggULolGjRti9e3e6L+BQ1KRJk1CyZEmsX78e//33HwwNDdG+fXtMnTpV7olZFeRkm8ydOxe1a9fG5s2bceTIEcTFxcHY2Bh2dnZo2rSpuJ62tjYOHjyInTt3Yt++fTh27Bji4uJgaGiIihUron///qhUqVJ2NzVPZPU4zS+ysv/Y2dnB3d0dDRs2lMq1V7NmTejq6iIqKkpuXz9w4EBoampi06ZN2LFjB4oUKYIWLVpg5syZ6V4AChecS5cuRffu3aGhwWEoUVZwbJ85HNsrz+LFi1GtWjVs27ZNfKqwYsWKmDJlCt95oIDM7DsGBgbYvn07/vnnH+zYsUN81F+ZgWwAWL16NaysrLB9+3Zs3rwZpUqVwvDhw+Hq6iozNqpfvz7Onj2LxYsX48KFC4iIiECJEiUwZMgQTJgwQe5LI9MzYsQIWFhYYN26dThw4ADi4+NhaWmJefPmYciQITJ5gN3c3DB16lScPn0aERERSE5ORr169bIVyAZSZmXv378ftWvXztNxLo8vWey/FVegQAEcPXoUK1euxKFDh7B+/XoULFgQtra2GDt2rEz6wZkzZ8LAwAB79uzB1q1bYWhoiA4dOmDy5Mky6wIpk+pOnz6NmTNn4vz587h8+TKsra1x6NAhXLp0KVcD2ZLw8PDkjFcjVbZ7924MHz4c69atQ48ePfK6OgpbuHAhFi1ahGPHjuX6SwOJiIgo93Xp0gVnzpzB9evXYWFhkdfVIVJJHNsTEeWeNWvWYPr06VizZg169eqV19UhogwwR/YvZPjw4dDX10f//v3zuirpcnBwgL6+vlJmXRIREVH+8OjRI5w5cwZNmzZlEJtIARzbExHlrNjYWGzatAn6+voKvW+IiPIen+n8BVStWhWTJk0S/63qj3337t0b9vb24r+z+ygQERERqS53d3c8ffoU+/btAwCpMQsRyeLYnogoZwUEBMDf3x/nz5/Hq1evMHPmTOjo6OR1tYhIAUwtQkREREQ5xsnJCQEBAShbtiwmTJjw2+Z5JCIiItUgpEIqVqwYunTpgjlz5vDdHUT5BAPZRERERERERERERKTSmCObiIiIiIiIiIiIiFQaA9lEREREREREREREpNIYyCYiIiIiIiIiIiIilcZANhERERERERERERGpNAayiYiIiIiIiIiIiEilaeR1BShto0aNQlRUVF5XQ2Vdv349r6ug8sqUKZPXVcgXEhIS8roK+cLXr1/zugr5goGBQV5XQeV9/Pgxr6uQL3AMIJ+BgQGuXLmS19UgyrTmzZsjPDw8r6uhsr59+5bXVVB5RkZGeV2FfCE6Ojqvq5AvLFiwIK+rkC/89ddfeV0FlSeRSPK6CvkCrxPl09fXx+nTpxVen4FsFRYVFYXIyMi8robK+vLlS15XQeXp6urmdRXyBQayFcNAtmI4kMsY+2/FcAxA9GsJDw9n/5cOBvkzpqmpmddVyBd4I1gx8fHxeV2FfCEsLCyvq6DyeP1DuYmpRYiIiIiIiIiIiIhIpTGQTUREREREREREREQqjYFsIiIiIiIiIiIiIlJpDGQTERERERERERERkUpjIJuIiIiIiIiIiIiIVBoD2URERERERERERESk0hjIJiIiIiIiIiIiIiKVxkA2EREREREREREREak0BrKJiIiIiIiIiIiISKUxkE1EREREREREREREKo2BbCIiIiIiIiIiIiJSaQxkExEREREREREREZFKYyCbiIiIiIiIiIiIiFQaA9lEREREREREREREpNIYyCYiIiIiIiIiIiIilcZANhERERERERERERGpNAayiYiIiIiIiIiIiEilMZBNRERERERERERERCqNgWwiIiIiIiIiIiIiUmkMZBMRERERERERERGRSmMgm4iIiIiIiIiIiIhUGgPZRERERERERERERKTSGMgmIiIiIiIiIiIiIpXGQDYRERERERERERERqTQGsomIiIiIiIiIiIhIpTGQ/QsrX7485s6dCw8PDxw5cgQrV65Eo0aNFP6+jY0NRo4ciQ0bNsDT0xNHjhzBhg0b0LVrV2hqasr9jpqaGlq0aIEVK1Zg//79OHToEDZt2oQ///wTBgYGyto0papZsybc3d0RGhqKt2/f4ty5c3B1dc1UGVpaWpg4cSJu3LiBDx8+4NGjR1i1ahWKFy+e5nc0NTUxfPhweHt74/Xr13j9+jUCAgKwdOnS7G6S0lWuXBnr16+Hv78/rly5gl27dqFly5YKf79MmTIYOnQoVq9ejXPnzuHu3bs4depUmusPHToUd+/eTffP7NmzlbFpSlWlShW4ubnhypUruH79Ovbt24dWrVop/P2yZcti+PDhWLduHby9vfHw4UOcO3cuw+9JJBK0b98eu3btwpUrV3Djxg2cPHkS8+fPh46OTnY2KUdUr14du3fvxpMnTxAcHIwTJ06gbdu2mSpDS0sLY8eORUBAAEJDQ3H79m0sXbo0zWOuSJEimDlzJgIDA/Hy5Uvcv38fW7ZsQYUKFZSxSUrHfUkxNWrUwIEDBxASEoLXr1/jzJkzcHFxyVQZWlpamDBhAoKCgvDu3Ts8ePAAK1askLsvrVu3Dl+/fk33z/jx45W0dcpjY2ODY8eO4fPnz/j27Rv8/f3RsWPHTJWhpaWFadOm4eHDh4iKisLLly+xYcMGGBoayqxrbW2N2bNnw9/fH2/fvkV0dDSePHmCNWvWoFSpUsraLKLfTo0aNbB37148f/4coaGhOHXqFNq1a5epMrS0tDB+/HhcvXoVr1+/xr1797B8+XK5fd6aNWvw+fPndP+MGzdOWZunNOzzFGNtbY0dO3bg/v37ePLkCY4ePQpnZ2eFv29qaoqxY8di27ZtCAoKEq9l0tOvXz8sW7YMZ8+eFc/d9evXz+6m5BhhnBEaGoo3b97g7NmzWRpnTJw4EdevX8f79+/x8OFDrFy5UqHrxIsXL+LVq1d49eoVLl++jCVLlmRzi3LGixcvsGzZMgwdOhQDBw7EnDlzcOXKlUyXExERgd27d2PixIkYMGAAhg0bhjlz5uD8+fNS63379g07duzA7NmzMXLkSPz5558YPXo0Fi1ahKCgICQnJytr05TKxsYGx48fR1hYGCIiIuDv749OnTplqgyhb3r06BGio6Px6tUrbNy4UW7fVLBgQYwZMwa7du3C/fv38ePHDyQmJsLU1FRZm6R0ud1/C220c+dO3Lt3D/Hx8UhISFDpNgJ4PS0JDw9XzaNciZycnODv74/w8PC8rkqm/Pnnn4iMjMzSd62trTF//nzEx8fDx8cH379/h62tLUqUKIFNmzbBw8Mj3e9ramri+PHjiI+Px507dxAcHAwtLS3UqlULZcqUwePHjzFhwgTExcVJfW/69Olo2LAh3rx5g6CgIPz48QMVK1ZElSpVEBYWhuHDh+Pr169Z2qafBQYGZrsMOzs7eHp6IjY2Fp6enoiMjETbtm1hamqKv//+G2vXrs2wDIlEgoMHD8LBwQFXr16Fv78/LC0t4ezsjNDQUNjb2yMsLEzqO/r6+vDw8ICNjQ0CAwNx7do1ACkDQltbW1hYWGR72wDAxMQk22XUrl0bbm5uiIuLw6lTpxAdHQ0HBweULl0aS5YswY4dOzIso127dpg3bx4SEhIQHBwMCwsLvH//Ps3AnI2NDWrXri33sw4dOsDY2Bjjx4/H6dOns7VtgoSEhGyXUadOHWzevBnx8fE4ceIEoqOj0aJFC5QuXRqLFi3Cf//9l2EZLi4uWLhwIRISEvDixQtYWlri/fv3cHBwSPM7mpqaWLVqFZo2bYpHjx7h6tWriI+PR8mSJVG3bl107NgRHz58yPb2AcCXL1+yXYatrS327duHuLg4HD58GFFRUXBycoKJiQlmzpyJjRs3ZliGRCLBnj170KxZMwQFBSEgIADm5uZwdHTEy5cv4ejoKHXMGRgYwMvLC5aWlrh27RqCgoJgbGwMJycnJCYmokOHDrhx40a2t01QtGjRbH3/d9iX3r9/n+0yGjZsCA8PD7H/joqKQtu2bWFiYoJp06Zh3bp1GZYhkUhw4MABsf++fPkyLCwsxP67efPmUvuSo6MjqlatKresESNGQFdXF/b29krbn7I6BkitSZMmOHHiBGJjY7F//35ERUXB1dUVZmZmmDBhAlasWJFhGRKJBMePH0fLli0RGBiIS5cuwcrKCi4uLggODoatrS0+f/4sru/v74+6devi6tWruHLlCuLj41GnTh3Y2dnh06dPaNKkCR4/fpzlbSpWrBiePHmS5e9T/pdfx/a1a9fO8rm0YcOGOHDgAOLi4nDo0CFERUXB2dkZJiYmmDFjBtavX59hGRKJBPv27YO9vT2uXbsm9nlOTk4IDQ1Fq1atpPq81q1bp9nnDRs2DLq6umjevDlu3ryZpW36mTL+P3/FPi+1EiVKKKWcBg0aYNeuXYiLi8PRo0cRFRUFR0dHlC1bFnPmzMGmTZsyLKNTp05YsWIFEhIS8OzZM5QrVw5v375NNzD9+vVrACnjAIlEAmNjY3Tq1CnDAHhmRUVFZbsMOzu7dMcZil4nuru7p3md6ODgIHOdWKRIkXSvEy0tLbO9bYLVq1dnu4yHDx9iyZIl0NTURN26dVGwYEEEBQXh8+fP6Nq1K1q3bq1QOaGhoViyZAliYmJgbW2NkiVLIi4uDm/fvoWGhobUTbPg4GD8888/sLS0hJGREQoVKoSIiAjcunULERERaNy4Mfr375/tbRP069cv22U0adIEJ0+eFPumyMhItG/fHmZmZhg/frzCfZOXlxdatmyJgIAAXLp0CeXKlRP7pgYNGkj1Taampnjx4gUAICQkBHp6eihWrBgsLCwQGhqa7W36uW7ZlRf9t6mpKZ4/fw5Auo0sLS2V3kZAyhg2u37F62kDA4NM3fzKVCA7NDQU1tbWsLe3TzMQ6uvrizZt2qBfv34K7Wi5IbuD3Tdv3qBy5cpYuHAhhg4dCgCwt7eHnp4eDh8+LPc7ERER+Oeff3D06FF8/PgRxsbGcHFxwaRJk6Crq6vQ72Y1kK2mpoatW7eiePHiGDVqlNh56ejoYPXq1TA2Nsaff/6Jjx8/plmGuro6OnfujGPHjkkNBNTV1TF9+nTUr18fmzdvxsGDB8XPKlSogNWrV+PRo0cYO3YsEhMTxc+GDBkCV1dX7NixA7t37870NsmT3UC2uro6goKCUKpUKTg4OODu3bsAgMKFC+PChQswMTFBrVq18OrVq3TL6dGjB9avXw93d3cMGDBAXN6/f3+sWLEC//77L0aPHi31nV27dsHJyQkDBw6UakOhXqnbLjuyG8hWV1fH0aNHYWxsjB49eoiDcF1dXezZswelS5eGs7Mz3r17l245ZcqUQdGiRfH48WPExcWJg5vMzDAFUjr+s2fPIioqCs2aNVNKABrIfiBbXV0dXl5eKFGiBLp27YpHjx4BSGmnAwcOoHTp0mjdujXevn2bbjllypRBsWLF8OjRI8TFxeHWrVv4/PlzusHHyZMno0+fPli2bBm2bNki9ZkwoFDWrITsBrLV1dXh7++PkiVLwtHREffv3wcA6Onp4dSpUyhbtiwaNGggXuSkpWvXrli1ahU8PT3FPhkAevfuLd5cmTBhgrh8wYIF+PPPP7FhwwbMmjVLXG5jY4MjR47g2bNnaNKkidLaKTuB7N9lX8puIFtdXR1Xr15FqVKl0Lx5c9y7dw9ASv997tw5mJiYoHbt2hn23927d8e6detw8OBBDBw4UFzer18/LF++HP/99x/GjBmTYX2sra3h7e2N+/fvo2HDhtnattSyG8hWV1fH/fv3UaZMGdja2uL27dsAUtopICAAZmZm+OOPP/Dy5ct0y+nTpw+2bt2KvXv3olevXuLyQYMGYf369di0aROGDRsmLh8+fDhOnTolXhgIJkyYgIULF2Zp1khqDGQrD8f2uTu2z2ogW11dHQEBAShVqhRatWol9nl6eno4e/YsypYti7p162Z4/uzWrRvWrFkDDw8PDB48WFzet29fLF26FNu3b1dohrW1tTXOnz+P+/fvo3HjxpnenrRkN5D9q/Z5qSkjkK2urg4fHx+UKFECbdu2xYMHDwCk7E/Hjx9HmTJl0KhRI7x58ybdckxMTFC8eHE8ePAAsbGxePbsGT59+pRuILtZs2a4e/cuPn36hIULF6JXr14qGchWV1fHtWvXxHFG6uvE8+fPw8TEBDY2NgpdJ65btw7u7u4y4wzhOvHnccauXbvg6OiIQYMG5eh1IpD9QHZiYiImT56Mr1+/Yvr06eIs1piYGMyePRufP3/GokWL0p19DgDfv3/H33//jfj4eEycOFHmGjYxMRHq6urivxMSEqCmpgY1NTWZcubMmYO3b99i/vz5KFOmTLa2T5DdQLa6ujoePHiAMmXKoEGDBlJ9U2BgIMzMzFCxYsUM+6a+ffuKfVPPnj3F5YMHDxb7ptTXRoUKFUL9+vVx/fp1fP36FSdOnEDLli1VMpCdV/33z20k3ChQ1UD2r3o9ndlANlOLKMDX1xcAxLQc3759w61bt9JM0xEdHQ0nJyesX78e5cuXx7Bhw1CuXDmsWbMGbdu2RWxsbI7Wt3r16ihVqhQuXrwoBrGBlBPKvn37oKWlhebNm6dbRmJiIvbu3SszCEhMTMT+/fsBANWqVZP6TBhY3bx5U+YEe/XqVQApd5hVRePGjWFhYQF3d3dxcAKkXKgsW7YM2tra6NatW4bl9OnTBwBkUl1s27YNwcHB6NSpEwoUKCAut7GxQZs2bbB//36ZwQkApQ5OsqtOnTowMTHBiRMnpGaSREVFYcuWLdDS0lLokdbXr1/jzp07MjP4M6tt27bi0wLKCmIrQ926dWFqaorjx4+LgUcgpZ3c3NygpaWl0GOIr1+/xu3btxVuJyMjI3Tv3h1BQUEygUcgJeioSo/WNWzYEObm5jh06JB40gVSgnWrVq2CtrY2OnfunGE5wsBt/vz5Ust37NiBkJAQdOjQQeqYa9WqFRITE7F48WKp9YOCgnDmzBlUrFgRDRo0yM6mKQ33JcU0atQIFhYWOHjwoBjQAVL67+XLl0NbWxtdu3bNsByh/54zZ47U8n///RfBwcHo2LGj1L6UFmGgvHPnzsxsRo5r1qwZrKyssHfvXvGCAPhfQE5bWxu9e/fOsBzhJu3ff/8ttXzTpk14/vw5unfvLtVO69atkwnoAMCyZcsQExOTqTRnRDkhv43t7ezsYGFhAQ8PD6k+LzIyEitWrFC4zxP6qrlz50ot/++//xAcHCxz/kxLjx49AEBpk1OUhX2eYmxtbWFmZobDhw+LQWwgZX9as2YNtLW1FUp38PLlS9y4cSNT+/+FCxfw6dOnLNU7N6UeZ/x8nSiMMxS5ThT2t7TGGfKuE52dnXHgwAGVv04EgAcPHuDjx4+oV6+eVCoGHR0dtGnTBgkJCfDz88uwnPPnzyMsLAydO3eWOxErdRAbADQ0NGSC2EBKmgjhKZL0Ju3lNmX1TX/++ScAYOrUqVLL3dzc5PZN0dHROHfunNKeiM9JedV/56c2Ang9LWAgWwF+fn4oVqwYKlWqBCDl8bHExETY2dnJXX/VqlW4e/cuRo8eDU9PT8yaNQuenp4YPXo0bty4odCjf9lhbW0NALh+/brMZ8KytB4TVIQQQPz5RCrcsapRo4bMyaZOnToAoLRHD5VBmDV38eJFmc+EPFwZzazT1taGjY0Nnjx5IveO/MWLF6Grq4saNWqIy9q3bw8AOHz4MIoWLYqePXti7Nix6Ny5s8rlERfSe1y+fFnmM39/fwBArVq1cq0+Qu7yjFLj5DZh/06vnWxsbJT+uy1btoSmpiZOnToFHR0dODs7Y+DAgWjfvj2MjIyU/nvZZWtrCwDw9vaW+Uw4DjM6AWpra6NmzZp4+vSp3DvNPj4+KFSokNgPAilB2i9fviAmJkZmfeGuvjJn0WYH9yXFpNd/X7hwAcD/9re0aGtro1atWmn2397e3jL9tzwFChRAx44dxccgVYkwU/Ls2bMyn505cwYAMgywaGtro06dOnj06JHcWTDnzp2Drq6uQvtlcnIyfvz4oVI3Iun3lN/G9so6f9aqVSvd86euri6qV6+ebjkFChRAhw4dEBsbiwMHDii2AbmEfZ5ihBnTly5dkvnMx8cHAFCvXr1crZOqEcYZwpgiNeE6UZFxhjKuE8eMGaOS14kAxEkXVapUkflMWKZIWp0rV65AIpHAxsYG7969w9mzZ+Hl5YUbN25k6viJj4/Hw4cPIZFIULp0aYW/l9PS65uEVJmK9E1169ZVSt+kilSt/1ZVvJ5OoZFrv4T/3eU9evQoQkJCoKWlBRsbG0yYMEHmEaRbt25h9+7d8PPzw5s3bxAfHw8LCwt06tQJI0aMkPuywYCAAMydOxe3bt2CtrY2mjRpIjPjQNF6pr5T7Ovri6pVqyI4OBgAcOrUKWhra6NIkSJ48eIFNDQ0xDuHycnJ2LlzJ3R1daWm4gMpj5dt2bIFO3bswNixYzNdL0UJLxWR9zjY169fERMTk62OXXjJ38+B8pCQEBw6dAiurq7YvHkzrl27hh8/fuCPP/5AuXLlsGPHDqU/NpYdQn4xeTMoPn78iMjIyAxzVZubm0NdXV1uGanLtrS0FLdduECwtLTEpk2bpGapR0ZG4q+//oKnp2emtycnCHfW5Z0IwsLCEB0dnWsvQqhZsybMzc1x+/btNNs7rwhtEBISIvPZ58+fc6ydKleuDCDlkauTJ09KBRzj4+OxfPlybN++Xem/m1Xm5uYAIPWkiODTp0+IiooS10mLmZkZ1NXVxf74Z0LZFhYW4uNJX758QfHixaGjoyNz8hX6bmXmG8wO7kuKEfrm9PrvjP5Phf5b3v6YumwLC4t0z13t2rUT81mqWr5eKysrAMCzZ89kPvvw4QMiIyPFddJiaWkJdXV1uWWkLtvKyirDWVcdOnRAkSJF4O7urkj1KR/g2D53xvZCnyevv/r48SOioqIyHLMK58+0+rzU58/00ve1adMGRYoUgaenJ/u8fNrnCWMteWMpRcdjvzplXidmdMyldZ3o5uYm9zrx0KFDmd6enCK8O8XY2FjmM319fRQoUCDDdHIJCQl4/fq1mCrp0KFDUk8BGhoaYtSoUShbtqzMdyMiInD+/HkkJycjIiICt2/fxpcvX+Di4iK3TnmlXLlyAICnT5/KfCb0TcI6acmobxLKLleunEKz4FWNqvXfqorX0ylybUb2169f0aJFCyxevBj6+vro168f2rZti1u3bqFNmzY4fvy41Prbt2/H8ePHUalSJfTt2xe9evVCcnIyZs+eLTdxv4+PD9q2bYvr16+jbdu26Nu3r/jSkswOso4ePYqaNWuKf0JDQ+Ht7S3+e8eOHYiLi0OdOnVQs2ZNODk5id99/vw53r17h7p166JQoUJS5RYqVAh169YV39CcU4TflXe3RFj+c90UZWNjA0dHR4SGhsp90d7GjRuxYcMGGBkZwcXFBZ06dUKVKlVw48YNlessChcuDCDlBChPZGSkuE52yki9HgDxbblz5syBl5cXrK2tYWJiggEDBiApKQlubm5iUCmvCTkf08rTGhUVpXBeyOwSZiio2mxsICUnFZB2Pr6oqChxHWUScjEPGzYMjx49grOzM2rVqoUhQ4bg69evmDx5cpqzy/KCMo45oR3TKkP4P0hdzoULF6Curo7x48dLrVuzZk0xzVJGv5tbuC8pJq/6b3mER/MUefFtbhMugL99+yb384iIiAxTfilSRur10lKmTBmsXLkSMTExmDlzZrrrUv7AsX3uje1Vsc9TtVRKAPs8RQnjiPTG9zkx1shPlHnMZbQvpS5HyCU9e/ZsnDhxAtbW1jA1NcXAgQNV7joR+F+8QUdHR+7nBQoUwPfv39MtIyoqCklJSYiKisKRI0fQpUsXrFmzBitWrEC7du3w+fNnrFy5EvHx8TLfjYiIwOHDh3HkyBFcvHgR3759Q5cuXRRKw5ebcqNvUrQPV1Wq1H+rMl5Pp8jSjOwXL15g4cKFcj9LK/n6xIkT8fDhQ6xevVoqt82nT5/QtGlTjB49Gg4ODmIelrFjx2Lp0qVSKSqSk5MxcuRI7Nq1C4GBgeIjT0lJSRg1ahQSEhJw4sQJcQZIcnIyBg0alOm74HZ2duLMM29vb/z7779YunQpDA0NER4ejlGjRqFHjx5o0aIFAOmOO/UMLnksLCxw/vx5PH/+XGkvH8gt5cuXx9SpUxETE4P58+fjx48fUp9LJBL89ddfaNKkCdavX4+AgADExcWhUqVKGDZsGFatWoWJEyf+9i9oEvJ5PXjwQCqxvru7OwoXLozly5djyJAhGDlyZF5VUeUUKlQIzZs3R3R0NE6dOpXX1VEZwr705csXjBo1SsxR6OPjg+nTp2PTpk3o16+fmAv0d7Vo0SI0bdoUw4cPh42NDa5fvw5jY2M4OzvjyZMnqFy5MpKSkvK6mnmK+1LWmJubo0GDBggJCZH7iDalKFq0KI4dOwYjIyP07dv3tx8HqCKO7X/PsX1mmZubo379+ggJCeH5IB3s8yir0rtO1NPTw/LlyzF48GD89ddfeVVFpRNmXyclJaFFixZo3bq1+Fn79u3x7t07XL16FdeuXZNJ51KmTBls374dSUlJCAsLQ2BgIDw8PPDs2TMMHz5cJt0pEWWNql1PZymQHRwcjEWLFim8flhYGDw9PdGoUSOZBO2GhoYYOXIkJk2aBG9vb7Rq1QoA5D46IpFIMGDAAOzatQve3t7iYDcgIAAhISFo1aqV1GOMEokE06dPh6enZ6ZejGBiYiJOjz937hyKFy+OP//8ExKJBCdOnACQ8oIoIZdpahnd5cnoDooyREdHA0j7zqiOjk6m3+Rcrlw5LFiwAMnJyZg6darcN7i2bNkSjo6OWL9+vdhOQEoC+Hnz5mHDhg3o168fpkyZkqnfziny7oKnpqenl+GMH0XKSL1e6r+fPHlSZv0TJ05g+fLlGeYnzC3CfpLWrAxdXd0c3ZcFrVu3ho6ODjw8PDK8q58XhDvgac1O19XVTfPOsDJ+NyAgQOZFO35+foiLi1OpWRvKOOYymm0g/B+k3i/fvXuHli1bYuLEiWjWrBlq1KiBt2/fYvHixXj16hU2bdqEz58/Z3ZzcgT3JcUosi9l1E5Z6b9/1rNnT6ipqWHXrl0Z1jkvCG2Q3pgko5fbKFJG6vV+VrRoUZw5cwaVK1fG8OHDsWfPHoXqTrmLY3vVHtvn1Zj1Z927d4eamprKHsfs8xQjnPPTG9/nxFgjP1HmMZeZvkP4u7yJOydPnsTy5cszfHdHbhLiDWk9CR4bG5tmTOLnMgDI3bYaNWrg6tWrCAkJSTMvuZqaGgwNDdGmTRuoq6tj//798Pb2hr29vaKbkqNyo29SpA9XZarQf+cHvJ5OkaXUIvb29ggPD5f759ixYzLr37hxA4mJiYiPj8fChQtl/gQFBQGQzhkUHx+PtWvXolmzZihbtiwMDAygr6+PJk2aAIBUriXh7d3ykpqbmJhkKx+0n58fGjRoAIlEAiDlZTA6OjoqdQL52du3bwFA7nYbGBhAR0dHbv7stJQrVw4LFy6Empoapk6dmuasAuHFgKnfMit48eKFQjlLc1Pq/NU/MzIygp6eXpo5zQQhISFITExMc7vk5VcT9nN5HaiwrGDBggpsQc4TbljIe3t0sWLFUKhQIbk3NZRNSCuiKrnDfya0gZmZmcxnxYsXz7F2EvIoyxuwJCcnIzo6Wuptw3lNyMMlb1aboaEhdHV108zVJQgNDUViYmKaub/SyiP6/v17jB07FtWrV0fZsmVRt25drF27VsxHJ6/fygvclxSTOq/kz4T+O6Nc+kL/ndYsS6HstM4Dampq6Nq1KxISElQyUAFI5wL8mbGxMfT09NLMIyh48eIFEhMT08xLmF5OQyGgU716dfz111/YvHlzZjeBcgnH9qo9tk+dr/JnRkZG0NXVzXDMKpw/05tZnvq3fsY+D1Jl5+c+TxhryRtLKToe+9Up8zoxo2Mus9eJqjQeE/JQC7myUwsPD0dsbCxKlCiRbhna2triiyzlBb2FZfJSi8gjvGRSeBGlKkidv/pnQt8kL392ahn1Tenl4c4P8rr/zi94PZ0iV3JkC3dOAgMDsWjRIpk/wuOBwkxiAOjduzemTZuGiIgIuLq6YuzYsZg0aRKGDBkCAIiLixPXFS68hZxSP0v90qqM+Pr6ioPwOXPmIDg4GF+/fhWXHT9+HEWKFMHSpUuxcOFC7N69W+r7WcmFpWx37twBANSqVUvmM2HZ3bt3FSordRD777//TvetwxoaKRP85d0B09TURMGCBWXSkeQlf39/AEDTpk1lPhPu3maU1zs2NhbXr19H+fLl5c40atq0KaKionDz5k1xmfD4ecWKFWXWF5al9RhvbhMuROVdSAp3xH9+6aeylStXDlWrVsXTp0/FfVvVXLt2DUD67SS0pTIJL2OSd7I2MDBA0aJFM3XTKqddvnwZAMSgRWrCcSisk5bY2FjcvHkT5cqVk/sId+PGjREdHa3QiVRNTQ0uLi748eOHTC7XvMJ9STHp9d/NmjWTWictGfXfTZo0kem/U2vRogVKlSqF8+fP4927d5ndhFwhnG+E3HWpCSkUMkqJEhsbi2vXrqFixYpyb2o6ODggKipKZr/8OaCzcePGrG4GqSCO7f8nN8b2yjp/3rhxI93zZ1RUFG7duiX3+82bN0fJkiVx4cKFDF/ellfY5ylGOOc3atRI5rPGjRtLrfO7EsYQwpgiNeE6MbvjDHnXiULKngoVKsisLyxTletE4H/XrsLNx9SEZfK25Wd//PEHAMgdawqT9NI6H/xMOD+pUlqR9Pqmli1bSq2TltjYWFy9ejXTfVN+kZf9d37C6+n//93c+BHhMYcRI0akOdsjPDwckydPBpAyy+PUqVOwt7fHlStXsHr1akyfPh1TpkxBhw4dZMoXBo5pTWX/+PGjwnX18/MTB+HLly8HkHJCEZaFhobi3bt34r9/npGQ0Qyu9GaRKcvNmzfx9u1bNG3aVOpOjY6ODrp27Yr4+HicO3dOXF60aFGULVtW5g6olZUVFi5cCHV1dUybNg0PHz5M93fv378PAOjWrZvMm+d79uwJDQ0NlZn1CKTkSAwODkanTp1QtWpVcXnhwoUxbtw4xMXFYd++feJyY2NjlCtXTuZC5b///gMAmRe59O/fH+bm5nB3d5d6VP/IkSP4/PkzOnXqhEqVKonLNTU1xbQrqvI26itXruDVq1dwdHSUGoTo6upiwIABiI+Px9GjR8XlxYsXh7m5uVJfAOnq6gpAddpEnsDAQLx8+RLOzs5SNyh0dXUxePBgxMfH4/Dhw+JyQ0NDpbTTtWvX8OzZM9SvX18m8DlmzBgA8h9NzCu+vr4ICQmBq6urVJoKPT09jBo1CnFxcVJ5T42MjGBlZSXz6Kvwgqm///5bannv3r1hZmYGDw8PqWNOQ0NDZvaKRCLBrFmzUK5cOWzdulXuTJK8wH1JMT4+PggODkbHjh3FmTdASv89duxYhftvIWfujBkzpJb369cP5ubmOHjwoEyqFYEqv/BMIOTt7datG6ytrcXlhQsXxuTJkxEXFydV/xIlSqBChQoy7STMKpw/f77U8kGDBsHS0hJ79uyRaicDAwOcPn0a1atXx+jRo7F+/fqc2DzKQxzb/09ujO0vXbqE4OBgdOjQQarP09PTw5gxYxAXF4f9+/eLy42NjeWeP4WX0k6fPl1qed++fWFubi5z/kytR48eAKCyqZQA9nmK8vPzQ0hICFxcXKSuRfT09DBy5EjExcXh4MGD4nIjIyNYWlr+Vi+ATD3O+Pk6UZnjjKxcJ6YeB+a1SpUqwdDQEIGBgVJPDMbExODYsWPQ0NCQSgcSHh6Ot2/fyqQiEW4YeHl5Sd0ADQ8Px5kzZyCRSMSnv4GUYH5CQoJMfaKiosR9N3UfkNeU1Tdt2bIFALBgwQKp5YMHD5bbN+UnedV/5ze8nv7/3w4PD09WdOXQ0FBYW1vD3t4eHh4ectfx9fVFmzZt0K9fP6xYsQJAyqMmFStWRJ06dXD69OkMf8fDwwN//vknFi1ahMGDB0t9tnr1asyYMQPdunXDhg0bAKTcDXVyckKrVq2kTihASidXo0YNJCYmZvoN52PHjsWRI0fw7NkzSCQSnD59Gl26dMHJkyel8vWllpycjEqVKiEyMhKPHz+Wert5dHQ0KlSogGLFiikU0P3zzz/TfJt0RqytrTF//nzEx8fDx8cH379/h62tLUqUKIFNmzZJ/f+NGzcOLVq0wNKlS3H27FkAKQfCv//+Cz09PVy7dk3uoznR0dFSwUUdHR2sWrUKJiYmeP/+PYKCgsS8qhUrVhRfpqOsWRzKmClgZ2cHT09PxMbGwtPTE5GRkWjbti1MTU3x999/Y+3ateK669evR48ePTB06FCpixyJRIKDBw/CwcEBV69ehb+/PywsLNCmTRu8fPkSzZo1Q1hYmNTvOjk5YceOHYiLi8ORI0cQHh6OJk2aoFKlSjh9+jS6du2qlGT58u5EZlbt2rXh5uaGuLg4nDp1CtHR0XBwcEDp0qWxZMkS8aIIAObNm4d27dph2rRpOHLkiLhcX19f6g23zs7OiIuLE/c3AFi6dKnMMaqhoYELFy6gUKFCaNasWY7ls5I3EMqsOnXqYPPmzYiPj8eJEycQHR2NFi1aoHTp0li0aJF4wwNIGXy4urpiypQpUoNRfX19TJw4Ufx327ZtERsbizNnzojLFi9eLNVO1apVw7///gtNTU2cO3cO79+/R82aNWFtbY379++jV69eSssr/uXLl2yXYWtri3379iEuLg6HDx9GVFQUnJycYGJigpkzZ0rNYFq1ahW6du2Kv/76S+oCXSKRYM+ePWjWrBmCgoIQEBAAMzMzODk54dWrV2jdurXUMVeyZElcunQJ3t7eePnyJbS0tNCkSROUL18eZ8+eRf/+/RV+XFERRYsWzdb3f4d9SRnngYYNG4qDLE9PT0RFRaFt27YwMTHBtGnTsG7dOnHddevWoXv37hg2bBj27t0rLpdIJDhw4IDYf1++fBnm5uZi/+3g4CDTfwMpNxDu37+PL1++oHLlypnK1ZsZWR0DpNakSROcOHECsbGx2L9/P6KiouDq6gozMzNMmDBBHKsBwNatW9GnTx/0799fqm+XSCQ4fvw4WrZsicDAQFy6dAmWlpZwdXVFSEgIGjRoIBVwPH/+PBo3boyHDx+m+VK+VatWZblPL1asGF+epiQc2+fu2L527dpZPpc2bNgQBw4cQFxcHA4dOoSoqCg4OzvDxMQEM2bMkAqerlmzBt26dcOIESOk2k8ikWDfvn2wt7fHtWvXxD7P2dkZL1++RMuWLdPs8+7cuYMvX76gWrVqOdbnZfb/U55fsc9LLaM0DYpq0KABdu3ahbi4OBw9ehRRUVFwdHRE2bJlMWfOHGzatElcd/ny5ejcuTPGjBkjtX0GBgZSN0U6dOiA2NhYeHl5icvmzp0rldd2+PDh4k2fWrVqwdLSEt7e3uLNqdOnTyvUp2Qks++DksfOzi7dccbP14nCOOPn60R3d/c0rxPt7e3lXidu375d/L8JDw9H48aNxevEbt26Ke2laqtXr852GQ8fPsSSJUugqamJunXromDBgggKCsLnz5/RtWtXqZc3bt68GX5+fhgwYADs7Oykytm7dy9OnTqFokWLokaNGkhISMDNmzcRERGBjh07ok2bNlLl3Lp1C+XKlUOxYsWgpaWFz58/4/bt24iLi0Pt2rUxfPhwMYVUdvXr1y/bZTRp0gQnT54U+6bIyEi0b98eZmZmGD9+vFTftG3bNrFvEm6GACn7k5eXF1q2bImAgABcunQJVlZWYt9Uv359mRvAixcvFmezC9fwHh4e4jGydevWDJ8uUIQy2jov+m8gpY2KFSsGQH4bbdu2TSltBED8nez4Fa+nDQwMcOXKFYXXz9LLHjPL2NgYrq6u8PT0xOrVqzFy5EiZHT0oKAiVKlWCjo6O+OhNYGCg1GD34cOH4kyK1OrXrw9TU1OcPn0aAQEBUm82nzt3bpYHW/7+/qhXr55Y18uXL6NgwYJyU3YIJBIJevXqhcWLF2PJkiWYNWuW+NmSJUsQFRWFsWPHZqk+mXH79m2MGzcOvXr1QuPGjaGuro6QkBBs27YNPj4+GX5fR0dHvGtTu3ZtqTuggvfv30sFsmNiYjBq1Ch06dIF9evXR/PmzaGmpobPnz/j+PHj2Lt3r8q8UE3g6+uLli1bYurUqXB1dYWmpiYePHiAWbNmKZyPOTk5Gd26dcOYMWPQtWtXDBs2DF+/fsXOnTsxd+5cuRcEXl5ecHR0xIQJE+Do6IiCBQvi+fPnmDFjBtauXZurb3zNyLVr19C7d28MHz4cLVu2hIaGBp4+fYoVK1YoPNDU0dFBu3bt0l22fv16mQsYe3t7GBgY4NSpUyr/UoarV6+iZ8+eGDFiBFq3bg0NDQ08efIEy5Ytk/tiT3l0dHTEGeiCQoUKSS1bu3atVDvduXMHXbp0wYgRI1CvXj0UKlQI7969g5ubGzZt2qRyL8f09/dH27ZtMWHCBLRr1w4aGhp49OgR5s2bJ3XzIz3Jycno06cPRo4ciU6dOmHQoEEIDw/Hnj178M8//8gccxERETh16hTq1KmD5s2bIyEhAY8ePcLYsWOxZ88e8W3pqoL7kmL8/PzQunVrTJkyBe3bt4eGhobYfyv6BEdycjJ69OiB0aNHo0uXLhg6dKjYf8+fP19u/w3878mjffv25VhAR1m8vb3RuHFjzJw5E507d4ampibu3buHKVOmpBlw+VlycjJcXV0xadIk9OjRA6NGjcKXL1+wbds2zJgxQ+bcbmpqCiDlUeGfZ6EJtm/frvL9OqWNY/sUuTm29/Pzg5OTEyZNmgQXFxdoaGjg4cOHmDNnjsIzNJOTk9GrVy+MGjUKnTp1wpAhQxAeHo7du3djwYIFafZ5Xbt2haamJvbv388+7xfp8y5fvoz27dtj3LhxaNOmDTQ1NfHo0SMsWLBAbl58eQoVKoTOnTunu2z58uVSgewmTZrI3DRK/Yj869evlRLIVgZfX1+0atUKU6ZMkbpOnDlzZqbGGd27d8eYMWPQpUsXqevEefPmpXmd6OTkhPHjx6N169ZS14nr1q1TqetEIGW///vvv3Ho0CFcvXoViYmJKFOmDDp37oy6desqXE63bt1QpkwZnD9/Hr6+vpBIJDA1NUWfPn1gY2MjtW6DBg2QlJSEFy9e4OHDh/jx4wd0dXVRoUIFNGzYMFO/m1u8vb3RqFEjzJo1S+yb7t69iylTpuDAgQMKlZGcnAwXFxdMmjQJPXv2xOjRo8W+afr06XJjLR06dJB5907qJ6F8fHyUFqTNrrzovwGINxRSU9U2Ang9DeTSjGwgJVdR27ZtcffuXVSqVAl16tRBkSJF8ObNG9y8eRPPnz/H48ePYWxsjMTERLRo0QLXr19H/fr1Ubt2bbx+/RonT55EixYtcOTIEalZG0DKTt+pUyeoqanB1dVVvGPw4cMHFClSBPfv38/UXf5Pnz6hXLlymD9/PoYPHw4gJV9PgQIFMjy5R0dHo2XLlrh37x6aNWsGa2tr3L59GxcuXEDNmjXh5eWl0Mv8sjMj+3fwu+duU4QyZmT/DpQxI/t3oIwZ2b+D7M7I/h2oan5VVcMxgHycka08HNvn7tg+OzOyfwfKmJH9q1PWjOxfnTJmZP8OlDEj+3egjBnZvzplzX7/1SljRvavKLMzsnMlRzaQUrEzZ85gzpw50NLSgru7OzZt2iQmY9+4caP4n6quro79+/ejZ8+eCAkJwaZNm/Do0SPMnTsXs2fPllt+kyZNcOTIEdSqVQtHjhzBf//9h7Jly+LkyZPQ19fPdH2FBOlCTqeYmBjcunVLKsdTWgoVKgQvLy8MHToUT548wdq1a/HkyROMGDECR44cUWigS0RERESkqji259ieiIiIKLdlakY25S7OyE4fZ2RnjDOyFcMZ2YrhLDLFcEZ2xjgjWzEcA8jHGdmUX3FGdvo4IztjnJGtGM7IVgxnZCuGM7IzxhnZiuGMbPlUdkY2EREREREREREREVFWMJBNRERERERERERERCqNgWwiIiIiIiIiIiIiUmkMZBMRERERERERERGRSmMgm4iIiIiIiIiIiIhUGgPZRERERERERERERKTSGMgmIiIiIiIiIiIiIpXGQDYRERERERERERERqTQGsomIiIiIiIiIiIhIpTGQTUREREREREREREQqjYFsIiIiIiIiIiIiIlJpDGQTERERERERERERkUpjIJuIiIiIiIiIiIiIVBoD2URERERERERERESk0hjIJiIiIiIiIiIiIiKVxkA2EREREREREREREak0BrKJiIiIiIiIiIiISKUxkE1EREREREREREREKo2BbCIiIiIiIiIiIiJSaQxkExEREREREREREZFKYyCbiIiIiIiIiIiIiFQaA9lEREREREREREREpNIYyCYiIiIiIiIiIiIilcZANhERERERERERERGpNAayiYiIiIiIiIiIiEilaeR1BShturq6eV0FlVa0aNG8roLK09fXz+sq5AsJCQl5XYV8ITk5Oa+rkC/wuMtYfHx8XlchX9DS0srrKqgkAwODvK4CUZbw/JA+dXX1vK6CymP/pxiePxXDdlJMsWLF8roKKk8ikeR1FfIF9uHyZXZ8JAkPD2dkgoiIiIiIiIiIiIhUFlOLEBEREREREREREZFKYyCbiIiIiIiIiIiIiFQaA9lEREREREREREREpNIYyCYiIiIiIiIiIiIilcZANhERERERERERERGpNAayiYiIiIiIiIiIiEilMZBNRERERERERERERCqNgWwiIiIiIiIiIiIiUmkMZBMRERERERERERGRSmMgm4iIiIiIiIiIiIhUGgPZRERERERERERERKTSGMgmIiIiIiIiIiIiIpXGQDYRERERERERERERqTQGsomIiIiIiIiIiIhIpTGQTUREREREREREREQqjYFsIiIiIiIiIiIiIlJpDGQTERERERERERERkUrTyOsKUNpq166NL1++5HU1KB+TSCR5XYV8oWjRonldhXwhLCwsr6uQL9y/fz+vq6DyKleunNdVoHysaNGiuHbtWl5XgyjTOLZPn7a2dl5XQeXZ2trmdRXyhaCgoLyuQr4wePDgvK5CvuDm5pbXVVB5P378yOsq5AtJSUl5XQWVZGBgAH9/f4XXZyBbhX358oWBI8oWBrJJmdgfkbJwXyKi3xHH9uljIDtjkZGReV2FfIE3jBTz/fv3vK5CvsD9KWPx8fF5XYV8gYFs5WBqESIiIiIiIiIiIiJSaQxkExEREREREREREZFKYyCbiIiIiIiIiIiIiFQaA9lEREREREREREREpNIYyCYiIiIiIiIiIiIilcZANhERERERERERERGpNAayiYiIiIiIiIiIiEilMZBNRERERERERERERCqNgWwiIiIiIiIiIiIiUmkMZBMRERERERERERGRSmMgm4iIiIiIiIiIiIhUGgPZRERERERERERERKTSGMgmIiIiIiIiIiIiIpXGQDYRERERERERERERqTQGsomIiIiIiIiIiIhIpTGQTUREREREREREREQqjYFsIiIiIiIiIiIiIlJpDGQTERERERERERERkUpjIJuIiIiIiIiIiIiIVBoD2URERERERERERESk0hjIJiIiIiIiIiIiIiKVxkA2EREREREREREREak0BrKJiIiIiIiIiIiISKUxkE1EREREREREREREKo2BbCIiIiIiIiIiIiJSaQxkExEREREREREREZFKYyD7F2ZjYwMvLy98/foVUVFRCAgIQKdOnTJVhpaWFqZPn44nT57g+/fvePPmDdzc3GBoaJjmd7p3744rV64gKioKX758wbFjx1CjRo3sbk6OYTtlzMbGBsePH8eXL18QGRmJy5cvZ7mNHj9+jJiYGLx+/VqhNgoMDERkZCTCwsJw9OhRlW0jAKhevTr27t2LZ8+eISQkBCdPnkS7du0yVYaWlhbGjRuHK1eu4NWrV7h79y6WLVuG4sWLy12/QIECGDJkCM6fP4+nT5/i2bNnuHjxIsaMGQM9PT1lbJbS8ZjL2M2bN9G9e3eUL18e5ubmcHR0xJEjRzJVxpMnTzBs2DBUrVoVJiYmqFWrFqZNm4avX7/KXX/Lli0YNWoUmjZtitKlS6NEiRLw9/dXxubkGO5LimE7Ef0aeCwrplatWjh8+DDev3+PsLAw+Pj4oEOHDpkqQ0tLC1OnTsW9e/cQHh6OFy9eYN26dXLbqVy5cli/fj0CAwPx6tUrfPv2DY8fP4anpyeaNm2qrM1SuvLly2POnDlwd3fHoUOHsGLFCtjZ2Sn8fRsbG4wYMQLr16/HwYMHcejQIaxbtw5dunSBpqam3O+oqamhRYsWWL58Ofbt2wcPDw+4ubmhf//+MDAwUNamKU2NGjVw4MABhIaG4s2bNzh79ixcXFwyVYaWlhYmTpyI69ev4/3793j48CFWrlyZ5tgeADQ1NTF8+HBcvHgRr169wqtXr3D58mUsWbIkm1uUM0qVKoUePXpg8uTJmDp1KgYMGIDKlSsr/P3q1atj1qxZaf4xMzPLkd/NbdyfMlarVi0cOnQIb9++xadPn+Dt7Y327dtnqgwtLS1MmTIFd+7cwZcvX/D8+XOsXbs23fNcaocOHUJMTAy+fPmSlU3IFb/7eU4SHh6enOu/msucnJzg7++P8PDwvK5KplhaWiIsLCxL323SpAlOnz6N2NhY7Nu3D5GRkejQoQPMzMwwbtw4LF++PMMyJBIJTpw4gVatWiEgIAA+Pj4oV64cXF1dERwcjHr16uHz589S35k6dSrmz5+PkJAQeHh4QE9PD127doWWlhbs7e1x+fLlLG1PTvnV20kikWS7jCZNmuDUqVOIjY3F/v37ERkZifbt28PMzAzjx49XuI28vLzENrp06RKsrKzENqpfv77cNpo3bx5CQkLg6ekJPT09dOnSBVpaWnBwcFDqvlSsWLFsl2Fra4sDBw4gLi4Ohw4dQlRUFJydnWFiYoIZM2Zgw4YNGZYhkUiwb98+NGvWDNeuXUNAQAAsLCzg6OiI0NBQtG7dWqpP0NDQwLFjx2BjY4O7d++KQceGDRuiSpUqePjwIVq2bInv379ne/sAyPwfZcWvfswBwPv377P1fT8/P3Tr1g3a2tpwcXFBoUKF4OXlhdevX2PmzJkYOnRohmVcv34dnTp1QmxsLFq1agVTU1Pcv38fly5dgqWlJY4dO4aiRYtKfadEiRIAAGNjYyQnJ+Pjx4/w8PCAra1ttrZHHuG3suN32JeU4Vdsp2LFiuH58+dZ/j7lfxzb/xrH8s+0tbWzXUbjxo1x7NgxxMbGwt3dHZGRkXB1dYWpqSkmTZqEVatWZViGRCLBkSNH0KJFCwQGBsLX1xdWVlZo164dQkJC0KhRI6l2cnV1xdq1a3HlyhW8fPkSkZGRKFWqFJydnVGkSBHMnDkTixYtyva2AVBawKBatWqYN28efvz4AR8fH8TExKBhw4YwNjbG5s2b4enpme73NTU1cfToUcTHx+POnTsICQmBlpYWatasiTJlyuDx48eYNGkS4uLipL73999/o2HDhnjz5g2uX7+OHz9+oGLFiqhcuTLCwsIwcuTING+4Z0ZAQEC2y7Czs4OHhwdiY2Ph6emJqKgotG3bFiYmJpg2bRrWrl2bYRkSiQTu7u5wcHDA1atX4e/vD0tLSzg7OyM0NBQODg4yfUKRIkXg4eEBGxsbBAYG4tq1awAAU1NT2NrawtLSMtvbJhg9enS2yzAzM0OvXr2QkJCAe/fuIS4uDpUqVYK+vj5Onz6t0P9F9erV4eLigkePHskdR9+6dUumv1fG7ypq5cqV2S7jV9+f4uPjs11Go0aNcPToUcTGxuLgwYOIjIyEi4sLTE1NMXnyZKxevTrDMiQSCQ4fPozmzZvjypUrYv/dtm1bhISEoEmTJule0/br1w+rV69GfHw8JBKJzPVSdiUlJWW7jF/xPFesWDE8ePBA4fUzFcgODQ2FtbU17O3t4eHhIXcdX19ftGnTBv369cOKFSsUrkhOyu5g982bN6hcuTIWLlwoBhDs7e2hp6eHw4cPy6x/584dHD58GLdu3cLt27cRFhYGW1tbeHl5Zep3szrYVVdXx6NHj1CmTBnUq1cPt2/fBgAULlwYV69ehZmZGcqXL4+XL1+mW07fvn3x77//Ys+ePejRo4e4fPDgwdi4cSPc3NwwZMgQcbmVlRUePHiAFy9eoE6dOoiIiAAAWFtbIzAwEC9evECVKlWQnKwa905+h3bKbiBbXV0dDx8+RJkyZVC/fn2pNrpy5QrMzMxQoUIFhdpo27Zt2LNnD3r27CkuHzx4MDZs2IBNmzbJtNH9+/fx4sUL1K1bV6qNAgIC8OLFC1StWlVp+1J2A9nq6uoICAhAyZIl0bp1a9y7dw8AoKenhzNnzqBs2bKoV68eXr9+nW453bp1w+rVq+Hh4SHVHn369MHSpUuxfft2jB8/Xlzerl07bNmyBcePH0e/fv2kytq+fTscHR0xfPhwHDhwIFvbJ8huIPt3OOaA7AWyExIS0LBhQ7x79w5eXl6oUqUKACAiIgKtW7fGq1ev4O/vj7Jly6ZbTpMmTfDo0SNs374dLVu2FJevW7cOc+fORe/evbF48WKp75w9exbW1tYwMjLCxIkTsWPHDpUNZP8u+1J2/artxEC28nBsz7E9oDp9XnYD2erq6rhz5w5Kly6NRo0a4c6dOwBS2snPzw+mpqaoWrVqhu3Uu3dvbNq0Cfv370efPn3E5QMGDMDatWuxZcsWjBgxQlyupaUlN4hTsmRJBAYGwsDAAGXLlsW3b9+ytX2AcgLZampq2Lx5M4oXL44xY8bgxYsXAAAdHR2sWrUKxsbGGDBgAD5+/JhmGerq6ujUqROOHz+OqKgoqeXTpk1DvXr1sGXLFql+pXz58li1ahUePXqE8ePHIzExUfxs8ODBcHFxwc6dO7Fnz55sb2N2g5jq6uq4du0aSpUqhebNm+Pu3bsAUval8+fPw8TEBDY2Nnj16lW65fTo0QPr1q2Du7s7Bg4cKC4X+tN///0XY8aMkfrOrl274OjoiEGDBuHgwYMy9UrdbtmV3UC2mpoaRowYgcKFC2PLli3iGFhbWxsDBw6Evr4+1qxZk+G+LwSyhX42t35XUdkNZP8O+1N2A9nq6uq4desWSpcujSZNmkj135cuXYKpqSmqVauWYRv16tULbm5u2L9/v9T18YABA7B69Wps3boVI0eOlPtdExMTXL16FVu3bkX79u1hbGyscoHsX/U8l9lANlOLKMDX1xdAyh0iAPj27Rtu3bol/vtnXl5eWL58Ofz8/GBsbJxr9RQ0a9YMVlZW2LNnjzjQBVICIQsWLIC2trbUzpoWoXOcMmWK1HI3Nzc8f/4cPXr0QIECBcTl/fr1g6amJubPny8OdAHg9u3b2Lt3LypVqoSGDRtmd/OUhu2UMaGN9u7dK9NGCxcuVLiNBgwYACBlVk9qQht1795dbhstWLBA5dsISLnDbm5uDk9PTzGIDQCRkZFYuXIltLW10aVLlwzLEYL88+bNk1q+fft2BAcHo2PHjlLtZGpqCgA4f/68TFlnz54FoJzZ5srCYy5jfn5+CAkJgaurqxjEBlIGJ3/99Rfi4+MzvDEREhKCR48eoXr16lJBbAAYOnQoihYtCnd3d0RHR0t91rx5cxgZGSlvY3IQ9yXFsJ2I5OPY/tc8lps2bQpLS0vs379fvLgHUtpp8eLF0NbWlppQkRYh+DF9+nSp5Vu2bMGLFy/QtWtXqXZKK4Dz7t07BAQEQEtLCyYmJlnZpBxRvXp1lCpVCt7e3mIQGwBiYmKwf/9+aGpqwsHBId0yEhMTsW/fPqkgtrB8//79AICqVatKfVayZEkAKbNrfw6eXb16FUDK7FFV0KhRI1hYWODgwYNi0BFI2ZeWL18ObW1tdOvWLcNyevfuDQCYM2eO1PJ///0XwcHB6NSpk9S+ZGNjA2dnZxw4cEAm6AhAqUFsZTA3N0fRokVx9+5dqYkccXFx8PX1hYaGBqpXr/7L/G5WcX/KWJMmTWBpaYkDBw7I9N9LlizJdP89Y8YMqeVC/92lSxepNkpt48aNeP/+PebOnZuNLclZPM+lYCBbAX5+fihWrBgqVaoEAPD390diYmKaOcRcXFzg7e2NN2/e4NChQ7lZVQApnQAAnDlzRuaz06dPA0h5HCE92traqFu3Lh49eiT3bs7Zs2ehq6sLGxsbpf5ubmI7ZUyRuqZ10SfIqI3OnTsn00bC9sv7XWGZqrQRAHHG6sWLF2U+E5Y1aNAg3TK0tbVRq1YtPH36VO7MbR8fHxQqVEhqUPbw4UMAKbPIfta8eXMkJSWpVI5jHnMZEx7RFuqcmjALK6NZRsIMKnmDCTU1NZQuXRrfv3/HjRs3slnbvMN9STFsJyL5OLb/NY9lYUx67tw5mc+EG/wZ5YDW1tZGnTp18PjxY7ntdP78eejq6qJWrVoZ1qdo0aKoXbs2oqOjERwcrMgm5AohwCxvHHD9+nWpdbJCCI79HCQLDQ0FkBJIV1dXl/qsTp06AKDQbNzcINyguXDhgsxnwgSSjJ5Y09bWho2NDZ48eSJ3FunFixehq6srlW9eyAV8+PBhFC1aFD179sSYMWPQuXNnlcwhLuSulveUlLBMmHijiBIlSqB+/fqwtbVF5cqVUbBgwVz53ZzG/Slj6fXfwrKMbpxqa2ujdu3aePz4sdw2unDhAnR1dVGzZk2Zz4YOHQo7OzsMGTIEsbGxWdmEXMHzXAqNXPslpMxOXLNmDY4ePSrm0bKxscGECRNQv359qXVv3bqF3bt3w8/PD2/evEF8fDwsLCzQqVMnjBgxQu4LJAICAjB37lzcunUL2traaNKkSZbupkRGRuLTp0/iv319fVG1alXxP+bUqVPQ1tZGkSJF8OLFC2hoaEgFDP74449M/6YylStXDgDw9OlTmc8+fPiAyMhIcZ20WFpaQl1dXW4ZqcsuV64c/Pz8xL9HRkbiw4cP6a6vKthOGbOysgKgmm0k1E0VWFhYAIDUrBbBx48fERUVJa6TFjMzM6irq8stI3XZFhYWCAwMBJBysvLy8oKzszMuXLgglSPbxMQE48aNk7pTm9d4zGVM+H82NzeX+czIyAiFChXKcJAgPAInb2CSlJSEN2/eAEgZ6GfmpU6qhPuSYthOlNM4ts8dPJYVI+R6ffbsmcxnQjtlNH60sLCAurq63DJSl21lZSUzWcDKygpdu3aFuro6SpYsCScnJ+jr62PkyJEyM5fzUunSpQFAHA+k9vXrV8TExKBUqVJZLr9FixYAZAPlISEhOHz4MFxcXODm5oagoCD8+PEDf/zxB6ysrLBz506l5jXODmFfkhco/fjxIyIjIzMc25ubmys0tre0tBS3W5iwYmlpCTc3N6kZ6pGRkfjrr7/y5GZaWoQxp7yUSVFRUYiLi8vU06H16tWT+vePHz/g7e0tc6wp+3dzGvenjKXXRpntv9NKP5e6/079fgdLS0vMmTNHfJmhKuN5LkWuBbK/fv0KR0dHPHz4EPXq1UO/fv0QGRmJEydOoE2bNvjvv//g7Owsrr99+3acOnUKDRo0QPPmzfH9+3f4+flh9uzZuHHjBnbu3ClVvo+PDzp27Ag1NTW4urqiZMmS8PHxQatWrTL9iNLRo0cxfPhwqWWhoaEyd26EO8dly5aVekQkrwnbm1Z+moiIiAzbRJEyUq8n/D2tXGry1s9rbKeMsY0Uo6enByBlQCBPZGQkChcunG4ZwuepH939uYzUvyXo168fpk2bhhEjRkjNntm3bx+8vb0Vqn9u4f6UMeH/Oa39RU9PL819RGBpaQlTU1PcunULZ8+eRfPmzcXPNm3aJL6BO6NyVBn3JcWwnSgncWyfe3gsK0aRbcxoPCaUkdF4TF455cqVw7Rp06TWHTRoEPbu3Ztx5XNRoUKFAEAmxZggJiZGXCezbGxs0Lp1a7x8+VLuTH43Nze8f/8ef/75J9q1aycuDwwMVKmnCBUZlys6ts/omEtdTvHixQEAs2fPhru7O/755x+Eh4ejRYsWWLp0Kdzc3PDkyRPcv38/cxuUQ4TUAz+/1FMQFxenUO77r1+/4sSJE3j27BkiIiJQsGBBmJubw8HBAc2bN8ePHz/E9DPK/N3cwv0pYxltnzLaSF7/LZFIsHnzZrx//x6zZs3KbLVzHc9zKbIUyH7x4gUWLlwo97O0kopPnDgRDx8+xOrVq8XcPgDw6dMnNG3aFKNHj4aDg4PYKY0dOxZLly6VeuwoOTkZI0eOxK5duxAYGCjesUtKSsKoUaOQkJCAEydOiDNAkpOTMWjQILi7u2dq++zs7LB9+3YAgLe3N/79918sXboUhoaGCA8Px6hRo9CjRw/xbrOOjk6myiciUoaCBQvCzc0NtWrVwuDBg+Hj4wMg5THf+fPno1mzZmjVqlWGL8WgX4tEIsE///yD3r17o0+fPmjdujVMTU3x4MEDeHt7448//sDDhw+z/TJYIvp1cGzPsT1l38mTJ1GgQAFoamrC1NQU/fv3x9atW2FjY4Nx48bldfVyXPny5TFlyhTExMRg/vz5+PHjh9TnEokEI0eOROPGjbFhwwYEBgYiLi4OlSpVwpAhQ7BixQpMnjwZT548yaMtyHtqaimZXx88eCC+iBYA3N3doaenh+XLl2Pw4MH466+/8qqKOSI0NFRMPQOkBMfu3LmDd+/eYdCgQWjSpAmCgoKy/aK8383vuj9lxpgxY1CnTh20atUK379/z+vqqDxVOc9lKZAdHByMRYsWKbx+WFgYPD090ahRI6mBLgAYGhpi5MiRmDRpEry9vdGqVSsAKTMhfiaRSDBgwADs2rUL3t7e4mA3ICAAISEhaNWqldRjjBKJBNOnT4enp2emEtmbmJiIjxOeO3cOxYsXx59//gmJRIITJ04AAPr06SPO2lA1wt2ZtGZIFC5cGF+/fs12GanXE/6emfXzGtspY2wjxaQ1W1qgp6eH8PDwdMuQdxf95zJS/xaQ8qbx1q1bo2fPnmKuSiAlD1psbCx27tyJMWPGYOzYsQpvS07i/pQx4f85vTvk+vr6GZbTtGlTHDlyRHw52blz51ChQgX8+++/8PX1xcOHD8VZGvkR9yXFsJ1IURzbc2z/KxzLimxjRuMxoYyMxmPpPdX048cPPHv2DFOnToWOjg6GDx+O06dPy52hnBeEmdhpzbrW0dHJ9CPi5cqVw7x585CUlIRp06bJvQHWokULtG7dGhs2bMDJkyfF5UFBQZg/fz7Wr1+PPn364O+//87Ub+cERcblio7tMzqGUu9Lwt9PnTols/7JkyexfPlyqRzIeU3IJZzW7Gdtbe1s5Rv+9OkTXr58CUtLSxQvXlx8QiSnf1fZuD9lLKPtU0Yb/dx/W1lZYdq0adi0aZOYUkvV8TyXIksve7S3t0d4eLjcP8eOHZNZ/8aNG0hMTER8fDwWLlwo8ycoKAiAdN63+Ph4rF27Fs2aNUPZsmVhYGAAfX198aUjqd9Oe+/ePQDyX6ZmYmIi5gHLCj8/PzRo0ECcuebv7w8dHR2VOeDlSS9nnbGxMfT09NLMjyd48eIFEhMT08x7Jy9X39OnT6Gnpyf3be7p5fbLK2ynjAn5kVSxjdLK6ZQXUuev/pmRkRF0dXXTzGcmCA0NRWJiYpr50eTl4RZe8ijvxCssy87LepSNx1zGhP9neXmwP378iOjoaLn5s+WpWbMmdu3ahcePHyM0NBRnzpxB69atxZeEqtLb3DOL+5Ji2E6kKI7tObb/FY5lIS+qvPygQjtlNH4MDg5GYmJimjlGheWKjkOFF3Jl9HL03CTkxpZ3HBkYGEBHRwdv375VuLxy5cph/vz5UFNTw7Rp09KcUS28SFTe+1uCg4MRGRkp5n/Na8K+JK8+RkZG0NPTy3BsHxISotDYPnU+X+F4kneDSFgmPOWiCoR0dfLyUevq6kJbW1tuHuvMiImJAQBoaWnl6u8qE/enjKXXRpntv9PqR37uv//44w8UKFAAQ4YMQUxMjNQfU1NTFChQQPy3qqTR4nkuRZYC2ZklzBAIDAzEokWLZP4IjwemztPVu3dvTJs2DREREXB1dcXYsWMxadIkDBkyBIB0PiThTkFas8uMjIwUrquvr684CJ8zZw6Cg4Px9etXcdnx48dRpEgRLF26FAsXLsTu3bsz1xi5QEgvIDwemVrLli2l1klLbGwsrl69iooVK0q97EbQvHlzREVFiRcqyvrd3MR2ypgidb106VK6ZWTURg4ODjJtJJQp73eFZarSRgDEl0U0bdpU5jNhWeoXSsgTGxuLGzduoFy5cihTpozM540bN0Z0dLTU29yFF2PJ6/uEZfHx8YptRC7gMZcxYeahvPzmFy9elFonK169eoWrV6+ifPnyef7ysuzgvqQYthPlFI7tcxePZcUI40cHBweZz4T3Rfj6+qZbRmxsLK5du4YKFSrIbSd7e3tERUXh+vXrCtWpZMmSAICEhASF1s8NQv73n3PEA0CtWrWk1smIEMRWV1fHtGnT8Pjx4zTXFcat8gJCmpqaKFiwoEw6krwi5Otu1qyZzGfCRJKMcnrHxsbi+vXrKF++vNwnUpo2bYqoqCjcvHlTXCbsnxUqVJBZX1iWVrqnvBASEgJAfvBRWJY6ZUhmSSQS8cWjqWeZ5vTvKhv3p4wJdZXXfwvLMpo1HRsbi6CgIFSoUEFuGzVr1gxRUVHii2hDQ0Px33//yf0TGRmJhIQE8d9p5WPPbTzPpciVQLYwNX3EiBFpzvYIDw/H5MmTAaTM8jh16hTs7e1x5coVrF69GtOnT8eUKVPQoUMHmfKFKfGfP3+W+/tpvaREHj8/P3EQvnz5cgApO4KwLDQ0FO/evRP/vWfPnky1RW44f/48nj9/ju7du8Pa2lpcXrhwYUydOhVxcXHYsWOHuLxEiRKoUKGCzKMFmzZtAgCZnImDBw+GpaUldu/eLfXIzr///osfP37g77//lirL2toa3bp1w4MHD1TqkQ22U8aENurWrZtMG02ZMkXhNtq8eTMAYMGCBVLLhTbas2eP3DaaOnWqyrcRkHJCCQ4ORvv27VGlShVxuZ6eHkaPHo24uDgcOHBAXG5sbAwrKyuZVCTCi65Sv0ABSHnc2dzcHAcPHpRqJ+GlJxMmTJDKd6ympoaJEycCyPiEn5t4zGXMzs4OpqamOHTokDgjEUgJ6qxevRpaWlro1KmTuPzDhw94+vSpzKNf0dHRSE5OlloWERGBkSNHIjExUSUe280O7kuKYTtRTuHYPnfxWFbMxYsX8eLFC3Tp0gXVqlUTlxcuXBgTJ05EXFyc1I2KEiVKoHz58jLttG3bNgDA3LlzpZYPGDAAFhYW2Ldvn1Q7pTWb38TERByPpU4Bl9du3bqFd+/eoUmTJlKzO3V0dNClSxf8+PFDnGEHpMzSLlOmjEzueCsrKzGIPX36dDx69Cjd3xVeKNelSxcxqC3o0aMHNDQ05M7Wzgs+Pj4IDg5Gx44dpZ5uLFy4MMaOHYu4uDjs27dPXG5sbIxy5crJ7EtCbv4ZM2ZILe/Xrx/Mzc3h7u4utS8dOXIEnz9/RqdOnVCpUiVxuaamJqZMmQIgJYWgqggODsaXL19QtWpVlChRQlyura0NOzs7JCQk4Pbt2+JyXV1dFC9eXCYliBAIS00ikcDBwQHFihVDcHCwVLqbzP5uXuP+lDGh/+7cubNM/z1hwoRM999z5syRWi703/v37xfb6M6dOxg2bJjcP1++fEFCQoL4b1VJVcPzXIos5cjOrJo1a0IikeDatWsKrS88Ut2iRQupF8IAKTnzfiYEji5fviyTqP7ly5fi41OKmDJlinhQjx07FkeOHMGzZ88gkUhw+vRpdOnSBSdPnszWjLiclpiYiAEDBuD06dO4dOkS9u3bh8jISHTo0AFmZmYYN26c1B3KhQsXom/fvujbt6/YOQIpHWWXLl3QvXt3mJubw8fHB1ZWVmjfvj1evHghE2x7+vQpZs2ahfnz5+P27dvw8PCAnp4eunbtCgAYOHCgTGAlL7GdMpaYmIiBAwfi1KlT8PHxwf79+xEZGYn27dvDzMwM48ePl2qjBQsWoG/fvujXr59MG3Xu3Flso0uXLsHS0jLdNpo9ezbmzZuHW7duwdPTE3p6eujSpQsAYNCgQSrTRkBKO40ZMwYHDhzA0aNHcejQIURFRcHZ2RkmJiaYMWOG1AsXp02bhq5du2LkyJFSg5Z9+/bBxcUFHTp0gImJCQICAmBubg4nJyeEhITIXHiuXLkSrVu3Fk9kwsWknZ0dKlasiOfPn2P9+vW50wgK4DGXMQ0NDSxbtgzdunWDi4sLXFxcUKhQIXh5eeH169eYOXOm1J3z+fPn48CBA1i5cqW4PUBK3ruFCxfC1tYWJUqUwOfPn3H69GmEhYVh0qRJ4ky61NasWSM+gijcgV+7di32798PAGjdujVat26dk5uvMO5LimE7UU7h2D538VhWTGJiIoYOHYpjx47h3LlzcHd3R2RkJFxdXWFqaopJkyZJtdPcuXPRq1cvDBw4UJxMAKRMLOjYsSO6dOkCU1NT+Pr6wtLSEi4uLggODsasWbOkfnfRokWwsLDAtWvX8OrVKyQlJcHCwgItW7aEtrY2li9fLnc/zytJSUlYuXIl5s2bhyVLlsDHxwcxMTFo2LAhjI2NsXnzZqmbRf369UPz5s2xbNkyMcCtq6uLBQsWQE9PD9euXUONGjVkAh3R0dFSQTIvLy84ODigRo0a2LRpE65fvy6+7LFixYoIDw+X+n/IS4mJifjrr7/g4eEBLy8veHp6IioqCm3btoWJiYlMHvCZM2eie/fuGDZsmNTNsD179sDV1RWdOnWCqakp/P39YWFhgTZt2iAkJATz5s2T+t3IyEiMGjUK27dvx9mzZ3H06FGEh4ejcePGqFSpEk6fPq1ST40kJSXh6NGj6NWrF/r164d79+6J/6f6+vo4ffq01ExqBwcHVK9eHYcPH5Z6ynTw4MF4//49Pnz4gMjISBQsWBCmpqYoXrw4vn37hqNHj2brd/Ma96eMJSYmYtiwYTh69CjOnDmDgwcPIjIyEi4uLjA1NcXkyZOl2mj27Nno1asXBg0ahF27donLd+3aJfbfZmZmYv/drl07BAcHY/bs2XmxeUrD81yKXJmRbWxsDFdXV3EGhrwBT1BQkJj/SHgMIDAwUGqdhw8fijMpUqtfvz5MTU1x+vRpqcZLTk7G3LlzM/UymNT8/f1Rr149cabj5cuXUbBgQfGRK1Xm7e2Nhg0bwt/fH126dMHQoUPx4cMHdOnSRW4bypOcnIx27dph5syZMDQ0xJgxY2Bra4utW7eifv36cmfJLFiwAD169MCnT58wdOhQdO7cGb6+vmjQoEGGqRXyAtspY97e3rCzs4O/vz86d+6MIUOG4MOHD+jatWum2sjFxQWzZs2CoaEhRo8eDVtbW2zbtg0NGjRIs4169uyJT58+YciQIejUqRN8fX1ha2urcm0EpPQXzs7OuHr1KlxcXNC3b198+vQJAwcOxIYNGxQqIzk5Gb169cKiRYtQvHhxDB48GHXq1MHu3bvh6Ogok+vtzZs3sLe3x5YtW6CtrY3evXujV69eUFdXx5o1a9CyZUuVegkTwGNOEQ0bNsSRI0dQu3ZtHDlyBDt27IChoSE2btwo9cbx9Pzxxx+oVKkSfHx8sGHDBpw6dQo1a9bEwYMHMWbMGLnfuXDhAg4cOIADBw6I+dcuXrwoLks9Q1wVcF9SDNuJcgLH9rmPx7JifHx80KxZMwQEBKBjx44YNGgQPnz4gJ49e2LVqlUKlZGcnIyOHTti7ty5MDQ0xF9//YX69evjv//+Q+PGjWXaacOGDQgMDIS1tTX69++PYcOGoWbNmjhx4gScnZ0xderUnNjUbLlz5w7Gjx+P+/fvo1GjRnBychJT7nh6emb4fR0dHfHJjNq1a6Nnz54yf1xcXKS+ExMTgzFjxmDfvn2Ii4uDg4MD2rRpgyJFisDLywsjR46UypWf13x9fdGqVStcuXIFrq6u6N+/Pz5+/Ih+/fph7dq1CpWRnJyM7t27Y+HChShevDiGDRuGunXrYufOnWjevLncPM5eXl5wcnLC5cuX0bp1a/Tv3x9Ayizc7t27IykpSanbmV0hISHYtm0bXr58icqVK6N27dqIioqCu7u7woGty5cvIz4+HpaWlqhfvz6qVq2KhIQEcRwr72W2yvjd3MT9KWOXLl2Cg4MDAgMD0aFDBwwcOBAfP35Er169sHr1aoXKSE5ORqdOnTBv3jwUL14cI0eORP369bF9+3Y0bdo0zSe98hOe5wBJeHi4wrfRQ0NDYW1tDXt7e3h4eMhdx9fXF23atEG/fv2wYsUKcfnXr1/Rtm1b3L17F5UqVUKdOnVQpEgRvHnzBjdv3sTz58/x+PFjGBsbIzExES1atMD169dRv3591K5dG69fv8bJkyfRokULHDlyBN26dZMKDnl7e6NTp05QU1ODq6srSpYsiUuXLuHDhw8oUqQI7t+/n6m7cp8+fRJzfg0fPhxASs6ZAgUKyH3pTWpPnjwRtz02NhaHDh2CkZGRmP8IgEKBLUtLS5V6SQHlP6nTTVDa5L0ohGT9Cif+3KBKF2GqKvVjoESZVaxYMamXGVHWcWzPsb0q+TndAMmS9z4WkqWKgUxVNHr06LyuQr6wcuXKvK6CylOl9zKpMlW6caBKihUrhgcPHii8fq6kFgFScmudOXMGmzdvhqenJ9zd3ZGUlAQjIyNUqVIFEyZMEINJ6urq2L9/P2bNmoXz58/j5s2bsLCwwNy5c+Hg4IAjR47IlN+kSRMcOXIE8+bNw5EjR1CgQAE0btwY//33n/gSmcwQZhjY2toCSLmDfOvWLYwbNy7D73748AF79+6VWvbx40epZYrO0CQiIiIiUjUc23NsT0RERJTbMjUjm3IXZ21QdnFGtmI4I1sxnJGtGM7IzhhnZFN2cEY25Vcc26ePM7IzxhnZiuGMbMVwRrZiOCM7Y5yRrRjOyJYvszOycyVHNhERERERERERERFRVjGQTUREREREREREREQqjYFsIiIiIiIiIiIiIlJpDGQTERERERERERERkUpjIJuIiIiIiIiIiIiIVBoD2URERERERERERESk0hjIJiIiIiIiIiIiIiKVxkA2EREREREREREREak0BrKJiIiIiIiIiIiISKUxkE1EREREREREREREKo2BbCIiIiIiIiIiIiJSaQxkExEREREREREREZFKYyCbiIiIiIiIiIiIiFQaA9lEREREREREREREpNIYyCYiIiIiIiIiIiIilcZANhERERERERERERGpNAayiYiIiIiIiIiIiEilMZBNRERERERERERERCqNgWwiIiIiIiIiIiIiUmkMZBMRERERERERERGRSmMgm4iIiIiIiIiIiIhUGgPZRERERERERERERKTSGMgmIiIiIiIiIiIiIpXGQDYRERERERERERERqTSNvK4Apa1o0aJ5XQXK5yQSSV5XIV/gsaaY5OTkvK4C/SKKFSuW11WgfIx9NuVX3HfTp62tnddVUHl6enp5XYV8gceaYgoWLJjXVcgXuD9l7MePH3ldhXwhKSkpr6ugkgwMDDK1viQ8PJyRCSIiIiIiIiIiIiJSWUwtQkREREREREREREQqjYFsIiIiIiIiIiIiIlJpDGQTERERERERERERkUpjIJuIiIiIiIiIiIiIVBoD2URERERERERERESk0hjIJiIiIiIiIiIiIiKVxkA2EREREREREREREak0BrKJiIiIiIiIiIiISKUxkE1EREREREREREREKo2BbCIiIiIiIiIiIiJSaQxkExEREREREREREZFK+z9cMbMRIQNRlQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1600x200 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tokens = [['[CLS]'] + glove_tokenizer.tokenize(sent) for sent in sentences]\n",
    "fig = plot_attention(tokens, alphas)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dNXPXiSlPIpa"
   },
   "source": [
    "# Contextual Word Embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J5oqS0x7PIpa"
   },
   "source": [
    "## ELMo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "id": "5N8dJJJmPIpa"
   },
   "outputs": [],
   "source": [
    "watch1 = \"\"\"\n",
    "The Hatter was the first to break the silence. `What day of the month is it?' he said, turning to Alice:  he had taken his watch out of his pocket, and was looking at it uneasily, shaking it every now and then, and holding it to his ear.\n",
    "\"\"\"\n",
    "\n",
    "watch2 = \"\"\"\n",
    "Alice thought this a very curious thing, and she went nearer to watch them, and just as she came up to them she heard one of them say, `Look out now, Five!  Don't go splashing paint over me like that!\n",
    "\"\"\"\n",
    "\n",
    "sentences = [watch1, watch2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1M_qVWdNPIpa",
    "outputId": "4648318e-e61b-45d1-ce57-be51d8bf4709"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sentence[58]: \" The Hatter was the first to break the silence. `What day of the month is it?' he said, turning to Alice:  he had taken his watch out of his pocket, and was looking at it uneasily, shaking it every now and then, and holding it to his ear.\""
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from flair.data import Sentence\n",
    "\n",
    "flair_sentences = [Sentence(s) for s in sentences]\n",
    "flair_sentences[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "HsasoxvnPIpb",
    "outputId": "bdc76cc2-d654-4a43-b0ec-8db2ecb9102c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Token[31]: \"watch\""
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "flair_sentences[0].get_token(32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bdaPu5VNPIpb",
    "outputId": "65a9bfe2-1685-426b-b656-f52ef136f948"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Token[31]: \"watch\""
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "flair_sentences[0].tokens[31]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "g_iTjhAuPIpb",
    "outputId": "cf1ca11d-1b0c-4472-baa1-cde1cdb0da1c"
   },
   "outputs": [],
   "source": [
    "# UPDATED\n",
    "###########################################################\n",
    "# Unfortunately, ELMO embeddings were returned by flair\n",
    "# using the allennlp library that has been archived.\n",
    "# from flair.embeddings import ELMoEmbeddings\n",
    "# elmo = ELMoEmbeddings()\n",
    "\n",
    "# As replacement, we'll be using flair's own embeddings instead\n",
    "from flair.embeddings import FlairEmbeddings\n",
    "flair_emb = FlairEmbeddings('news-forward')\n",
    "###########################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wZOGM9xpPIpb",
    "outputId": "bbe612f8-e6f8-409c-ff11-0a8ff020a4c7"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Sentence[58]: \" The Hatter was the first to break the silence. `What day of the month is it?' he said, turning to Alice:  he had taken his watch out of his pocket, and was looking at it uneasily, shaking it every now and then, and holding it to his ear.\",\n",
       " Sentence[48]: \" Alice thought this a very curious thing, and she went nearer to watch them, and just as she came up to them she heard one of them say, `Look out now, Five!  Don't go splashing paint over me like that!\"]"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# UPDATED\n",
    "###########################################################\n",
    "# Instead of elmo, embeds sentences using flair embeddings\n",
    "# elmo.embed(flair_sentences)\n",
    "flair_emb.embed(flair_sentences)\n",
    "###########################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UdFVdgRZPIpc",
    "outputId": "9f217b62-5258-4bb5-80b1-50167528a771"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(Token[31]: \"watch\", Token[13]: \"watch\")"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "token_watch1 = flair_sentences[0].tokens[31]\n",
    "token_watch2 = flair_sentences[1].tokens[13]\n",
    "token_watch1, token_watch2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3XbrpwUfPIpc",
    "outputId": "17511113-351f-4ccb-dafb-ac6c3af8bc0a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([-0.0007, -0.0057,  0.0187,  ..., -0.0051, -0.0022,  0.0026],\n",
       "        device='cuda:0'),\n",
       " tensor([-0.0011, -0.0037,  0.1031,  ..., -0.0079, -0.0044,  0.0002],\n",
       "        device='cuda:0'))"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "token_watch1.embedding, token_watch2.embedding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ucsg8w21PIpd"
   },
   "source": [
    "### Where do ELMo Embeddings come from?\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/elmo_lstm.png?raw=1)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/elmo_embed.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "AehQ4275PIpd"
   },
   "outputs": [],
   "source": [
    "# UPDATED\n",
    "###########################################################\n",
    "# Unfortunately, this piece of code only makes sense for\n",
    "# ELMO embeddings. Flair embeddings are structured differently\n",
    "# and therefore this code is commented out.\n",
    "# token_watch1.embedding[0], token_watch1.embedding[512]\n",
    "###########################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "VFAgysccPIpd"
   },
   "outputs": [],
   "source": [
    "# UPDATED\n",
    "###########################################################\n",
    "# Unfortunately, this piece of code only makes sense for\n",
    "# ELMO embeddings. Flair embeddings are structured differently\n",
    "# and therefore this code is commented out.\n",
    "# (token_watch1.embedding[:1024] == token_watch2.embedding[:1024]).all()\n",
    "###########################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3hxX2iMHPIpd",
    "outputId": "f2a2ba04-428b-4d6a-9748-ffc44987e072",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(0.5003, device='cuda:0')"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similarity = nn.CosineSimilarity(dim=0, eps=1e-6)\n",
    "similarity(token_watch1.embedding, token_watch2.embedding)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "id": "jF_w6p-rPIpe"
   },
   "outputs": [],
   "source": [
    "def get_embeddings(embeddings, sentence):\n",
    "    sent = Sentence(sentence)\n",
    "    embeddings.embed(sent)\n",
    "    return torch.stack([token.embedding for token in sent.tokens]).float()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "kEU5nynEPIpe",
    "outputId": "3adbeb72-7c7a-4f0c-9936-99d1fb3be4a7",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-2.0933e-03,  5.0415e-04,  4.6944e-02,  ..., -4.4014e-04,\n",
       "         -3.9301e-02,  1.0601e-02],\n",
       "        [ 3.2224e-04,  1.4939e-03,  2.5947e-02,  ..., -9.2415e-04,\n",
       "         -1.4211e-02,  3.0942e-03],\n",
       "        [ 1.7407e-03, -1.8324e-04,  5.3375e-02,  ...,  4.5165e-04,\n",
       "          5.4484e-02,  9.7125e-03],\n",
       "        ...,\n",
       "        [ 2.2250e-03,  2.2138e-04,  4.4385e-02,  ..., -3.9766e-03,\n",
       "         -7.9449e-04,  2.2444e-02],\n",
       "        [-2.2455e-03,  9.7197e-04,  3.4624e-02,  ..., -7.1559e-05,\n",
       "          1.2647e-02,  1.3210e-02],\n",
       "        [ 1.0997e-03,  8.4726e-05,  1.5534e-01,  ..., -1.5737e-04,\n",
       "          9.5801e-04,  5.3785e-03]], device='cuda:0')"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# UPDATED\n",
    "###########################################################\n",
    "# Instead of elmo, embeds sentences using flair embeddings\n",
    "# get_embeddings(elmo, watch1)\n",
    "get_embeddings(flair_emb, watch1)\n",
    "###########################################################"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fX_xJposPIpe"
   },
   "source": [
    "## GloVe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "p7Y54X9_PIpe",
    "outputId": "6d284631-1096-449c-d81d-9d85100553f3"
   },
   "outputs": [],
   "source": [
    "from flair.embeddings import WordEmbeddings\n",
    "glove_embedding = WordEmbeddings('glove')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mibBx6mvPIpe",
    "outputId": "daaf25a4-14b5-4147-8311-65861baa0dd5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Sentence[58]: \" The Hatter was the first to break the silence. `What day of the month is it?' he said, turning to Alice:  he had taken his watch out of his pocket, and was looking at it uneasily, shaking it every now and then, and holding it to his ear.\",\n",
       " Sentence[48]: \" Alice thought this a very curious thing, and she went nearer to watch them, and just as she came up to them she heard one of them say, `Look out now, Five!  Don't go splashing paint over me like that!\"]"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_flair_sentences = [Sentence(s) for s in sentences]\n",
    "glove_embedding.embed(new_flair_sentences)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "WzaqGrPBPIpf",
    "outputId": "42cee0a9-5191-4876-9e4c-e59b78cc8d6b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(True, device='cuda:0')"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.all(new_flair_sentences[0].tokens[31].embedding == new_flair_sentences[1].tokens[13].embedding)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mIH-ZF35PIpf"
   },
   "source": [
    "## BERT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 287,
     "referenced_widgets": [
      "2de16307e11046418b98b90111b4009b",
      "9a78ed83139b4f26bf8282022e3c403e",
      "046484ba77494b55acb7e13ed55a3487",
      "12e1763b7a9b4ae78074cee4b5a409a3",
      "47fddc1c2c074d87a6050b2a26d859f1",
      "886bcbe9c1394d569f0c74fd7b54f0a5",
      "42d404e579f04988813895e1d0212f81",
      "b2610c7ea73e437083cd43ef671cc563",
      "514fd68e776d4e4aaeb4a90f51bc4736",
      "f601a3994de84cc9885036e8d1c5a8ee",
      "64cecc3cdf97483990e52c2b0311601f",
      "0dcaee584779404aa4fbbad506ddcd3a",
      "53a7239ac99e4fd889edf4be66091d35",
      "57b9fbf00fe34967b6f342524aae352b",
      "e6f9a976e59e42618a3c511f427acb88",
      "7ce5227fb27046dfb9f47c48aa863fb8",
      "32945d26f3504f99a1b2140cd83d121e",
      "dbf509c4b21142ec9131647ab164374a",
      "c34e1baa25134764a8ecaa869b6bc406",
      "305ec2194a844366910861a1c2b73201",
      "ed7ba59f9d504f6c841ebcd8386efdb9",
      "6a39130caf3f455d8f0f1d002a74bdb1",
      "29e05722f6d94e049658cace58933099",
      "e243128654cd4c44873240ecc4644a17",
      "120b3e46872b41cdb2be157882b67ca4",
      "9aba3a0e80c34c8790985250931253f1",
      "2e61bcbdaa8d4ec299cc29ae92c80adb",
      "14d54788f46d479eb0faa38af384cdc8",
      "a12d5562a5d84bca8378845a92411bc6",
      "d9cdf9667f6746f382f5f10f772b3144",
      "5e05b1fd6b084563bc6b22e5650c6c79",
      "8b7af1b3fb044d428f5a98dfb804a356",
      "0e0863081114481c964e57672c86c0b7",
      "090e8bfcf7ed4be6870d9fcb068100a3",
      "dc87398fce094a658da749f69feb4fc8",
      "4e07a852aa1b4cf1bb57b7c77515c133",
      "8e8a43aa4e11455484df7d141e6ad248",
      "3c644b03c6ca4f388efbfe3f9cd573a4",
      "cc9a28d85306494ea73be4bbe52e9d17",
      "cba3e1f715b24d03b8cf179efb41055b",
      "afffdf5c259f4f14aeb99b6c25016ac6",
      "ff2247ba8c714949a5e80f599f4f47b4",
      "c70600d1e4784ea7bf7e58b06ee47cc1",
      "6381a57dd82848689065de71dda55749",
      "2493c820244c4097a40125268920c512",
      "208deb72133c40a7a88e01add562fd60",
      "1ea9534a1322429eba100f7a28b3cbb2",
      "61d1ee6617f3473da9b5e40b1ac00fbf",
      "a74b725d47d149d497dcb57a8f4b8df0",
      "9fcc4a2ca21042848e5becbb152cb681",
      "1612ba093f8c455fb8149092cf366571",
      "70db9c1a40214b50a30862988d18e717",
      "e3941149c2a84b6ab6f61b6666e393af",
      "072fea3dc9ba40f0b6e781f8755a322f",
      "a1c3ad8b4f61434b9d41fb0ffda79991"
     ]
    },
    "id": "wppmv3qLPIpf",
    "outputId": "f63242b4-7e4b-4a29-8cd5-857dd799acc8"
   },
   "outputs": [],
   "source": [
    "from flair.embeddings import TransformerWordEmbeddings\n",
    "bert_flair = TransformerWordEmbeddings('bert-base-uncased', layers='-1')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IXKlNCfdPIpf",
    "outputId": "d8d300b6-57e5-4241-e583-101723b5c225"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.6554, -0.3799, -0.2842,  ...,  0.0601,  0.8865,  0.4760],\n",
       "        [-0.1459, -0.0204, -0.0615,  ..., -0.0257,  0.5052,  0.3324],\n",
       "        [-0.0436, -0.0400, -0.0135,  ...,  0.3920,  0.5231,  0.9067],\n",
       "        ...,\n",
       "        [-0.2582,  0.6933,  0.2688,  ..., -0.0325,  0.0772,  0.2187],\n",
       "        [-0.1868,  0.6398, -0.8127,  ..., -0.5957,  0.2793,  0.1880],\n",
       "        [-0.1021,  0.5222, -0.7142,  ...,  0.3644,  0.0600, -0.1419]],\n",
       "       device='cuda:0')"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "embed1 = get_embeddings(bert_flair, watch1)\n",
    "embed2 = get_embeddings(bert_flair, watch2)\n",
    "embed2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ZNZPqZPUPIpg",
    "outputId": "2daedc15-16b4-485b-8142-b91116ce17f3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([ 8.5760e-01,  3.5888e-01, -3.7825e-01, -8.3564e-01,  1.3528e+00,\n",
       "          1.0204e-01, -8.2618e-01,  3.1570e-01,  3.1091e-01, -4.2653e-01,\n",
       "          1.6632e-01,  7.2440e-02, -1.0276e+00,  4.4680e-01, -1.4549e-01,\n",
       "          1.8315e-01,  3.7484e-01, -4.0245e-01, -1.4190e-01, -5.1596e-01,\n",
       "          1.1984e+00,  6.8202e-01,  1.1028e+00, -4.3956e-02,  1.6932e-02,\n",
       "          7.4420e-01, -1.0604e-01,  3.8719e-01,  7.3662e-02,  2.2424e-01,\n",
       "         -5.0282e-02,  1.9586e-01,  1.0773e+00,  1.0341e+00,  4.4217e-01,\n",
       "         -6.5567e-01, -5.6308e-01, -1.1827e-01,  1.2862e-01,  4.6581e-01,\n",
       "         -7.1305e-01, -1.5682e-01, -1.3552e-01,  4.5852e-02,  8.2852e-03,\n",
       "          5.2762e-01,  1.6906e+00,  5.5388e-01, -7.3777e-02,  5.1504e-01,\n",
       "         -2.0619e-01, -2.9503e-03,  2.5389e-01, -1.5379e-01,  8.4900e-01,\n",
       "          6.5440e-01, -3.2925e-01, -1.1198e+00,  2.4251e-01,  1.5586e-01,\n",
       "          5.7514e-01,  8.5832e-02,  3.3306e-01, -3.9344e-01, -4.1766e-01,\n",
       "          1.0790e+00,  1.1719e-02,  7.0214e-01, -6.0665e-01,  7.3702e-01,\n",
       "          6.8223e-02, -3.7110e-01, -7.7589e-01, -1.3057e-01,  1.7179e-01,\n",
       "          6.2051e-01, -6.4163e-01,  4.2881e-01,  5.7445e-01,  3.9801e-02,\n",
       "         -2.1279e-01, -5.1984e-01,  7.5919e-01,  3.0600e-01, -3.3987e-01,\n",
       "         -3.0427e-01, -5.7486e-01, -5.2273e-02, -6.8957e-01,  7.3935e-01,\n",
       "         -1.6545e-01, -8.2786e-01,  3.4169e-02,  7.1547e-01, -5.6962e-01,\n",
       "         -7.8686e-01,  3.6913e-01, -1.9961e-02, -5.7287e-01, -2.6407e-01,\n",
       "          8.5581e-01, -6.1419e-02,  3.8184e-01, -9.1566e-03, -1.0273e+00,\n",
       "         -5.3303e-01,  4.4014e-01,  1.7875e-01,  9.1851e-01,  5.9216e-01,\n",
       "         -4.8853e-01, -3.9106e-01, -8.8284e-02, -7.1935e-01, -3.8647e-01,\n",
       "          9.9550e-02,  2.2983e-01, -1.5325e+00,  6.5409e-02, -1.0896e+00,\n",
       "          1.2108e-01, -1.9921e-01, -2.5195e-01,  1.4421e-01,  2.7531e-02,\n",
       "         -3.0479e-01, -3.8533e-01, -1.1540e-01, -8.3979e-01, -7.7467e-01,\n",
       "          6.5181e-01,  3.1980e-01,  4.7616e-01, -8.2181e-01, -4.4323e-01,\n",
       "          1.2445e-01, -2.9082e-01, -4.2669e-01, -6.8031e-01,  1.1736e+00,\n",
       "          1.0199e+00,  7.3959e-01, -6.1510e-01,  1.0691e+00,  5.6806e-01,\n",
       "          9.4541e-01, -9.8110e-01, -7.1101e-01,  3.6151e-01,  2.0443e-01,\n",
       "         -5.0706e-02, -2.8712e-01,  1.2332e-01,  3.9614e-01, -4.4268e-01,\n",
       "         -5.5477e-01,  2.8325e-01,  5.5901e-01, -3.8944e-01, -4.2003e-01,\n",
       "         -1.2470e-01, -9.5830e-01, -3.4775e-02,  1.0381e+00, -2.2607e-02,\n",
       "          4.7267e-01,  8.1840e-01,  9.0778e-01, -1.9271e-02,  9.3007e-03,\n",
       "         -2.9097e-01,  5.1579e-01,  6.5791e-01,  3.1353e-01,  4.7983e-01,\n",
       "          6.2282e-01,  1.4813e-01, -3.8600e-01,  1.8680e-01, -7.3365e-01,\n",
       "         -5.2868e-01, -3.5756e-01,  1.4778e+00,  1.7065e+00,  5.2843e-01,\n",
       "          4.3788e-01, -6.0783e-01, -2.0891e-01, -2.6378e-01, -3.5563e-01,\n",
       "         -1.9136e-01,  4.4895e-01, -2.1979e-01, -2.2114e-01, -6.3080e-01,\n",
       "         -1.1537e+00,  3.8770e-01, -6.5096e-01, -1.4381e-01, -3.3387e-01,\n",
       "         -4.8165e-01, -2.2851e-01,  4.4550e-01, -1.4106e-01, -1.9820e-01,\n",
       "          1.2682e-01,  5.7340e-01,  5.2960e-01,  3.8535e-01, -1.8421e-01,\n",
       "         -2.7734e-01,  2.1106e-02, -5.1269e-01,  1.2724e-01, -5.2737e-01,\n",
       "          1.9794e-01,  6.6517e-01, -4.9908e-01, -5.2315e-01, -3.3081e-01,\n",
       "         -6.9502e-01, -2.1738e-01, -9.1546e-01,  8.4446e-01, -8.6271e-01,\n",
       "          9.8701e-01, -5.3786e-01,  2.2177e-01, -7.9477e-01,  1.6505e-01,\n",
       "         -3.9847e-01, -2.9883e-01,  2.0877e-01,  2.4684e-01,  1.0068e-01,\n",
       "          1.6014e-01, -1.1922e+00, -1.5679e-01, -2.0235e-03, -5.7467e-01,\n",
       "          4.6138e-03, -2.6884e-01,  4.2084e-01, -7.2791e-03,  3.4696e-01,\n",
       "          5.3609e-01,  2.1878e-01,  3.3520e-01, -2.8570e-01, -5.0958e-01,\n",
       "         -3.9454e-01, -8.9165e-01,  1.2181e+00, -6.8443e-01,  1.9280e-01,\n",
       "         -2.1449e-01,  4.6196e-02,  5.2594e-01, -4.3565e-02,  2.4555e-01,\n",
       "          1.9387e+00,  1.5278e-01,  9.9872e-01,  4.8577e-01, -4.1082e-01,\n",
       "         -8.3016e-01,  2.7045e-01,  4.8655e-01, -2.4722e-01, -1.8512e-01,\n",
       "          2.0524e-01, -6.6171e-02,  2.3728e-01,  1.2404e+00, -2.6077e-01,\n",
       "         -1.6077e-01,  1.3507e-01,  1.1847e-01, -2.9235e-01,  5.5437e-01,\n",
       "         -1.8560e-01,  1.3344e+00, -3.6667e-01, -3.7478e-01, -5.5665e-02,\n",
       "         -3.4930e-01,  8.5271e-01,  1.5741e+00,  6.0562e-01,  4.3780e-01,\n",
       "         -1.0298e+00,  4.0719e-01, -8.2617e-02,  2.7545e-02, -4.9199e-01,\n",
       "          7.3797e-01,  2.4101e-01,  9.6176e-01,  6.4367e-01,  1.1653e-01,\n",
       "          9.4047e-01, -2.6082e-01,  1.0976e+00, -1.1490e-01, -9.5253e-01,\n",
       "          5.8644e-02, -2.3060e-01, -6.7886e-02, -3.0703e+00,  2.4538e-02,\n",
       "          2.7582e-01, -4.3797e-01, -7.8420e-02,  4.2382e-01,  1.0763e+00,\n",
       "          1.8140e-01, -1.0572e+00, -3.9120e-01, -9.3492e-02, -3.7970e-01,\n",
       "          5.3947e-02,  1.1576e+00,  8.4130e-01,  1.8007e-01, -5.2631e-01,\n",
       "         -2.0291e-01, -1.9092e-02,  6.1720e-01,  1.1918e-01, -1.4457e+00,\n",
       "         -2.5592e-01, -1.2389e-01,  2.3928e-01,  1.1959e+00, -5.4284e-01,\n",
       "         -8.6952e-01, -4.8750e-01,  2.3500e-01,  1.3830e-02, -2.6659e-01,\n",
       "          9.7897e-01, -8.5489e-01,  3.1946e-01, -4.9998e-01, -3.2241e-01,\n",
       "         -1.3864e-01,  1.2900e-01, -7.5765e-01, -4.5315e-01,  2.9473e-01,\n",
       "          8.7803e-02, -1.3806e-01,  4.5430e-01,  8.0650e-02, -4.1994e-01,\n",
       "          5.8679e-03, -7.0910e-02,  3.8368e-01, -1.3059e-01,  8.7690e-01,\n",
       "         -6.1343e-01, -6.1015e-01, -7.0236e-01,  3.7512e-01,  3.3982e-01,\n",
       "          8.0856e-01, -1.6385e-01, -1.0569e-01,  6.1967e-01, -5.8364e-02,\n",
       "          2.3294e-01, -2.7713e-01,  5.2177e-01, -1.7158e-01, -1.0602e+00,\n",
       "         -4.3479e-01, -6.6953e-02,  3.6794e-01, -2.3135e-01, -2.0400e-01,\n",
       "         -5.3282e-01, -1.2286e+00, -2.4328e-01, -2.8658e-01,  1.3142e+00,\n",
       "          3.1902e-01, -9.1673e-02, -6.9225e-01, -3.0026e-01, -9.0491e-01,\n",
       "          8.4392e-02,  2.1298e-01,  8.2160e-01, -6.7558e-01, -8.9888e-01,\n",
       "          3.5113e-01, -5.2840e-01, -3.1060e-02, -2.8948e-01,  5.1492e-01,\n",
       "         -2.2329e-02,  1.6455e-01, -1.3406e-01,  7.9522e-01, -1.0633e-01,\n",
       "          4.0473e-01,  3.6492e-01, -6.1566e-01, -5.1250e-01, -5.4742e-01,\n",
       "          3.2135e-02, -4.5447e-02,  5.3155e-02,  7.0305e-01, -1.5488e+00,\n",
       "          6.2800e-01, -4.1181e-01,  3.1977e-01,  3.9906e-01, -1.7447e-01,\n",
       "          1.0357e+00, -5.4064e-01,  2.1086e-01, -5.4963e-01, -3.4768e-02,\n",
       "         -1.6814e-01,  4.8931e-01,  1.6012e-01, -6.6077e-01, -6.7637e-02,\n",
       "          9.2604e-02, -1.1114e+00,  4.6336e-01, -3.5745e-01, -3.2055e-02,\n",
       "          3.4265e-01,  3.5538e-01,  6.7498e-01, -2.0282e-01,  2.1987e-02,\n",
       "         -1.0548e+00,  9.8446e-01, -5.6733e-01,  6.9380e-02, -4.7736e-01,\n",
       "         -9.9546e-01, -3.5254e-02,  4.6173e-01,  7.1046e-01,  3.3707e-01,\n",
       "         -8.1136e-02, -4.8355e-01,  5.4515e-01,  3.7669e-02,  5.2528e-01,\n",
       "          4.1853e-02, -1.2564e+00, -3.2226e-01, -8.2649e-01, -7.5457e-02,\n",
       "         -7.5999e-01, -1.8107e-01,  1.1781e+00,  7.7896e-01,  6.6844e-01,\n",
       "         -7.6050e-01, -4.5029e-01, -5.4186e-01,  2.7288e-03,  2.3229e-01,\n",
       "         -6.9320e-01, -4.4791e-02, -7.4108e-02,  9.3096e-01, -4.1993e-01,\n",
       "         -7.3685e-01,  6.3003e-02,  4.0879e-01, -7.2932e-01, -1.6402e-01,\n",
       "         -2.7400e-01,  7.5049e-01, -6.5255e-01, -4.3226e-01, -6.2632e-02,\n",
       "         -8.2434e-01, -9.1996e-01,  5.1996e-01,  3.3342e-01,  3.0549e-03,\n",
       "         -3.7381e-01,  8.1910e-01, -2.5230e-01, -8.2635e-01,  3.8007e-01,\n",
       "         -1.0642e+00, -7.9937e-01,  1.0933e-01, -2.3652e-01,  4.5540e-01,\n",
       "         -2.9788e-01, -1.5564e+00, -2.3166e-01, -1.1862e+00, -2.9165e-01,\n",
       "         -2.1934e-01,  1.1169e-01,  1.3712e-01, -5.1983e-01, -2.5509e-01,\n",
       "         -4.4754e-01, -4.7948e-01,  4.6833e-01, -1.8625e-01,  6.9716e-01,\n",
       "         -1.1866e-01,  6.1084e-01,  6.3114e-01, -8.4579e-01, -4.6913e-01,\n",
       "          9.2668e-02,  2.9336e-02,  2.5508e-01,  4.5543e-01,  1.8251e-01,\n",
       "         -1.1855e-01, -1.2805e-01,  8.3519e-02,  1.7325e-01,  6.1112e-01,\n",
       "         -3.9921e-01,  1.6798e-01, -8.4275e-02,  3.9789e-01,  9.2255e-01,\n",
       "          7.7275e-02, -4.0163e-01,  3.1146e-01, -3.0563e-01, -5.8679e-01,\n",
       "          4.1130e-01, -4.3300e-01, -1.2496e+00, -8.5653e-01, -2.3101e-01,\n",
       "         -1.2980e+00, -2.7850e-01,  1.3634e-01,  3.4896e-01,  4.5002e-01,\n",
       "         -6.9828e-01,  6.2787e-01, -2.2598e-01, -1.7731e-01, -3.2118e-02,\n",
       "          4.8136e-01,  5.0901e-01,  5.0199e-01,  3.6993e-01, -4.5981e-01,\n",
       "         -6.7486e-02, -8.6722e-01, -5.7723e-01, -3.2765e-01,  7.8836e-01,\n",
       "          4.0974e-01, -1.8077e-01, -8.4644e-01, -2.5588e-01,  1.0655e+00,\n",
       "         -3.7329e-01, -4.9543e-01, -2.4806e-01, -3.7839e-02,  9.1718e-02,\n",
       "         -1.5725e-01,  2.4216e-01,  6.5864e-02,  1.3641e+00, -1.2707e+00,\n",
       "          7.8174e-02, -1.0401e+00, -3.9388e-01, -2.5627e-01, -4.9158e-01,\n",
       "         -4.0819e-01,  7.3981e-01, -4.4651e-01, -3.1208e-01,  6.2193e-01,\n",
       "          1.1230e+00, -4.4226e-02, -1.2044e+00,  2.2500e-01, -9.1115e-02,\n",
       "          3.5365e-01, -3.7870e-01, -1.0870e-01,  3.2082e-01,  1.2314e-01,\n",
       "         -2.1645e-02,  1.1413e+00,  1.3573e-02, -9.9737e-02,  9.3759e-01,\n",
       "          5.0417e-02, -4.9671e-01,  9.0127e-01,  1.4178e-01,  2.0027e-01,\n",
       "          6.3084e-01, -2.0751e-02,  3.0619e-01,  4.5247e-01, -7.6788e-01,\n",
       "         -7.2624e-01, -5.2419e-01,  6.0564e-01, -1.9887e-02, -2.9045e-01,\n",
       "          4.0384e-01,  9.0467e-01, -9.9711e-01,  1.0205e-02,  1.7958e-01,\n",
       "          3.5786e-02, -2.3696e-01,  6.4812e-01,  1.5352e-01,  3.1153e-02,\n",
       "          1.3805e-01, -1.4471e-01, -9.9713e-01,  1.7288e-01,  1.8074e-01,\n",
       "         -2.8294e-02,  7.8877e-01,  3.7700e-01, -7.0563e-02,  1.0712e+00,\n",
       "         -4.2392e-01,  4.1656e-01, -6.8028e-01,  1.6911e-01,  2.6617e-01,\n",
       "         -1.1312e+00,  1.6047e-01, -4.5070e-01,  4.9935e-01,  3.4413e-01,\n",
       "          6.4540e-01, -2.6726e-01,  1.5033e-01,  4.2149e-01,  3.4961e-01,\n",
       "         -1.5505e-01,  1.5471e+00,  7.7871e-01, -2.7912e-01, -4.5703e-01,\n",
       "         -7.9149e-01, -5.7845e-01, -2.4879e-01, -6.0028e-01,  6.9574e-02,\n",
       "         -1.9024e-02,  2.0337e-01,  1.3373e+00,  1.7257e-01, -3.3319e-01,\n",
       "          1.4024e-01, -2.2573e-02, -2.2282e-01,  3.4986e-02, -1.4985e-01,\n",
       "          5.2162e-02,  1.5562e-01, -2.3856e-01, -7.4214e-02, -2.1547e-01,\n",
       "          7.6900e-01,  1.1663e-01,  5.3913e-02, -2.6870e-01, -2.3951e-01,\n",
       "          8.8452e-01,  3.2750e-01, -7.8228e-02,  4.5901e-01, -4.3255e-01,\n",
       "         -4.4316e-01, -3.2504e-01, -1.4550e+00, -8.6594e-02, -7.3577e-02,\n",
       "         -2.0595e-01, -6.1714e-01, -7.1308e-01,  8.2567e-01,  1.6622e-01,\n",
       "         -8.5806e-01, -6.5291e-01, -1.5518e-02, -6.2837e-01,  5.6276e-01,\n",
       "          9.6669e-02,  4.9524e-01,  3.4259e-01, -6.7065e-01, -4.1591e-01,\n",
       "         -3.3399e-01, -7.4808e-01,  3.2285e-01, -5.1326e-02,  4.1144e-01,\n",
       "          2.6481e-01, -9.0313e-01, -4.4464e-01,  6.6254e-02,  4.1635e-01,\n",
       "         -7.5472e-01,  1.2165e+00, -4.2961e-01,  6.3139e-01, -2.1192e-02,\n",
       "         -4.6616e-01, -1.4575e-01, -7.4167e-01, -2.7592e-01, -4.6199e-02,\n",
       "          4.5393e-01,  4.7207e-01, -1.7622e-01, -6.7738e-01,  3.7787e-01,\n",
       "          5.3374e-01,  3.6832e-01,  1.3724e-01, -1.1172e+00, -2.5947e-01,\n",
       "          1.1888e+00, -1.0117e-01, -4.8335e-01,  5.8897e-02,  1.6610e-01,\n",
       "          6.6885e-01, -9.0577e-01, -2.3182e-01,  5.9258e-01,  1.1081e-01,\n",
       "          1.8942e-01,  6.5041e-01,  4.6973e-01, -3.1615e-01, -5.9470e-01,\n",
       "         -5.1061e-01, -9.7604e-01,  6.0229e-01, -6.3250e-01, -4.2595e-01,\n",
       "         -5.7413e-01, -7.1769e-01,  9.2507e-01, -6.2844e-01,  1.8694e-01,\n",
       "          2.0768e-01,  1.1880e-01,  4.1445e-01], device='cuda:0'),\n",
       " tensor([-9.8449e-02,  1.4698e+00,  2.8573e-01, -3.9569e-01,  9.6890e-02,\n",
       "          8.6436e-01, -3.7401e-01,  2.1752e-01, -2.5937e-01, -4.2487e-01,\n",
       "         -6.1410e-02,  3.3272e-01, -1.9102e-01, -2.9803e-02, -1.7782e-01,\n",
       "          7.1223e-01, -3.6092e-01,  4.7531e-01,  1.8901e-01, -1.9464e-01,\n",
       "          5.5041e-01, -2.0075e-01, -4.9742e-01, -1.3700e-02,  6.5577e-02,\n",
       "          2.2761e-01,  7.6017e-01, -7.9264e-02,  2.7941e-01, -5.4700e-02,\n",
       "         -2.6595e-01,  5.5095e-01, -2.1662e-01,  1.4398e-01, -3.8731e-01,\n",
       "         -1.9042e-01,  1.3155e-01,  2.2447e-01,  2.9488e-01,  2.7523e-01,\n",
       "         -2.7458e-01, -5.0480e-01, -3.9631e-01,  3.0837e-01, -6.3027e-01,\n",
       "         -3.5416e-01,  1.1856e+00,  8.9831e-02,  4.4822e-01,  4.0406e-01,\n",
       "         -3.4759e-01,  5.6113e-01,  1.0385e+00,  1.5012e-01,  3.7633e-01,\n",
       "         -3.9084e-01,  1.0627e-01, -5.0344e-02,  2.2779e-01, -5.2758e-01,\n",
       "         -1.9242e-01,  1.8374e-02,  5.2802e-01,  1.6686e-01, -7.3694e-01,\n",
       "          4.7303e-01, -4.7016e-01,  1.6289e-01,  9.7774e-02,  1.2824e-01,\n",
       "         -2.8870e-01, -3.9116e-01, -1.8324e-01,  5.2460e-01, -1.9830e-01,\n",
       "          5.4971e-01,  4.1160e-02, -3.3559e-01,  2.1168e-02, -8.5710e-02,\n",
       "          1.3109e-01, -3.6348e-01,  2.7382e-01,  1.1263e+00, -2.5038e-01,\n",
       "         -6.4161e-01, -3.2357e-01, -1.3938e-01, -1.0110e+00,  7.3331e-01,\n",
       "          1.0102e-01,  1.5387e-01, -2.9724e-01,  2.5424e-01,  3.2296e-01,\n",
       "         -3.5495e-01,  1.2272e-01,  9.6730e-02,  2.1486e-02,  9.1426e-02,\n",
       "         -1.4177e-01,  5.2628e-01,  2.1227e-01, -1.7098e-01, -3.8352e-01,\n",
       "          1.6601e-01,  4.4298e-01,  6.3218e-03, -5.9510e-01,  3.5965e-01,\n",
       "         -7.1792e-01, -3.6388e-01, -7.0435e-02,  1.8896e-01,  5.4331e-03,\n",
       "          1.1766e-02,  1.0752e-01, -5.5095e-01, -6.7836e-02,  3.5240e-01,\n",
       "         -2.6229e-01,  3.7230e-01, -8.1172e-02,  4.8515e-01,  1.0485e-02,\n",
       "         -2.9454e-01, -3.1111e-01, -2.6310e-01,  6.3381e-02, -1.0854e-01,\n",
       "          1.3688e+00,  2.9465e-01,  2.0623e-01,  3.8317e-02,  5.4437e-02,\n",
       "         -1.5726e+00,  3.7239e-01, -6.0964e-01, -8.8024e-01,  2.4682e-01,\n",
       "          6.7070e-01, -5.6562e-01,  2.1737e-01,  8.4505e-02, -2.6882e-01,\n",
       "          1.4112e-01, -8.0701e-01, -1.1845e-01, -3.2562e-01,  7.8287e-01,\n",
       "         -1.8123e-01, -7.1260e-01,  2.0073e-01, -2.2902e-01, -1.7190e-01,\n",
       "          8.2328e-02,  3.2728e-01,  3.3844e-01, -4.3359e-01, -2.0109e-01,\n",
       "          5.2177e-01, -2.1286e-01, -5.4271e-01,  4.2771e-01, -2.8439e-02,\n",
       "          6.8534e-01,  3.0166e-01,  3.3880e-01, -7.2538e-01,  4.7892e-02,\n",
       "         -5.7754e-01,  4.3418e-01,  6.2832e-01,  1.1230e-01,  1.6312e-02,\n",
       "         -3.1558e-01,  3.0021e-01, -3.4793e-01, -4.2287e-01,  8.9215e-02,\n",
       "         -1.7020e-01,  5.5911e-01,  4.3507e-01,  1.0352e+00, -6.0532e-02,\n",
       "          6.4033e-02,  6.3265e-01,  5.8860e-01, -6.2739e-01, -3.6926e-01,\n",
       "         -1.3001e-01,  3.0337e-01, -7.3679e-01, -1.0968e+00, -4.9620e-01,\n",
       "         -2.4885e-02,  6.0554e-02, -2.8982e-01, -5.5470e-01,  9.2083e-02,\n",
       "         -3.5928e-01, -6.8506e-01,  7.0259e-01,  7.4057e-01,  4.1953e-02,\n",
       "         -2.2261e-01,  3.1581e-02, -5.7034e-01, -1.5258e-01, -1.9746e-01,\n",
       "         -5.7486e-01, -6.6855e-01, -5.3475e-01,  2.1482e-01, -8.8653e-01,\n",
       "          9.2318e-02, -2.2077e-02,  9.8558e-01, -8.5093e-01, -1.6057e-02,\n",
       "         -1.1169e-01, -2.8778e-01, -4.2301e-01,  7.7859e-02,  6.9498e-02,\n",
       "          4.2390e-01, -7.0422e-02,  7.7603e-02, -4.0021e-01, -2.4131e-01,\n",
       "          1.3328e-01, -1.1587e-01,  9.0171e-01, -3.4312e-01,  6.0788e-01,\n",
       "          3.1653e-01, -2.4758e-01,  4.5196e-02,  4.9370e-01,  3.8995e-01,\n",
       "         -3.4692e-01,  3.3056e-02,  2.0106e-01,  4.0234e-01,  1.2993e-01,\n",
       "          2.9119e-01,  9.2159e-01, -4.1758e-01,  1.5004e-02,  5.2865e-02,\n",
       "          6.0922e-01,  7.6300e-02,  2.6940e-01, -6.3987e-01,  1.7187e-01,\n",
       "         -1.4997e-01, -2.2630e-01, -5.7578e-01, -5.4688e-02, -1.4096e-01,\n",
       "         -1.8318e-01,  1.6508e-01,  5.1326e-01,  4.4380e-01, -2.8080e-01,\n",
       "         -4.5417e-01, -3.0488e-01,  4.8724e-01, -1.7721e-01, -2.2289e-01,\n",
       "          6.0853e-01, -2.2062e-04,  5.6217e-01,  1.1224e+00, -1.4180e+00,\n",
       "         -3.6117e-01,  1.5190e-01, -1.7284e-01, -1.7632e-02, -2.1651e-01,\n",
       "         -8.6195e-01,  4.2032e-02, -4.6597e-01,  7.2990e-02, -6.2120e-02,\n",
       "         -3.9781e-01,  3.5995e-01, -3.4646e-01, -3.0336e-01, -1.5005e-02,\n",
       "          3.3628e-01,  5.7878e-02, -3.3315e-01,  1.1097e-01,  3.0777e-02,\n",
       "         -7.1115e-01,  3.6039e-01,  1.6007e-01,  3.4098e-01, -5.3309e-01,\n",
       "          5.6319e-01,  2.1157e-01,  2.6252e-01,  4.2321e-01, -5.2220e-01,\n",
       "          2.4814e-01, -3.6991e-01, -5.4816e-01, -5.3290e+00,  1.0130e-01,\n",
       "         -2.4280e-01,  2.6960e-01,  6.2188e-01,  3.4420e-01,  2.2724e-01,\n",
       "          1.2104e-01, -3.0778e-01, -1.3375e-01,  1.2070e-01,  6.9458e-01,\n",
       "         -1.7730e-01,  6.3572e-01,  3.9321e-01,  1.0829e-03, -4.1992e-01,\n",
       "         -3.1437e-01,  4.6219e-01,  3.0782e-01, -2.4236e-01, -6.3911e-01,\n",
       "         -7.2800e-03, -2.1637e-02,  2.9671e-01,  8.6041e-01, -1.6743e-01,\n",
       "         -5.2726e-01, -7.3631e-01, -4.9699e-01, -4.0285e-02, -5.5160e-02,\n",
       "         -2.8419e-01,  3.4066e-01,  1.9508e-01, -3.1847e-01,  1.1065e-01,\n",
       "         -1.3114e+00, -6.5144e-02, -5.2608e-01, -2.2926e-01, -4.9479e-01,\n",
       "         -3.6531e-01, -1.5898e-01,  9.3911e-02,  1.2114e-01, -1.5592e-01,\n",
       "          1.9144e-01, -7.1318e-01, -1.7203e-01, -6.8007e-02,  4.2680e-01,\n",
       "          2.7249e-01, -8.1866e-02, -7.7034e-01,  4.4868e-02,  6.1248e-01,\n",
       "          6.4211e-01, -4.2664e-01, -4.1450e-01, -4.6396e-02, -5.7337e-01,\n",
       "          4.2681e-01, -4.0819e-02,  2.8339e-02, -3.5921e-01, -2.4894e-01,\n",
       "         -4.9875e-02,  5.8249e-01, -1.4033e-02, -3.2829e-01,  1.0102e-01,\n",
       "         -4.5791e-01, -1.0023e+00, -2.6306e-01, -2.5233e-01,  6.0858e-01,\n",
       "          2.0588e-01, -2.5912e-01, -1.0158e-01, -3.4643e-01, -1.6082e-01,\n",
       "         -3.1902e-01,  5.3895e-01,  2.7880e-01, -6.8466e-01, -7.6329e-02,\n",
       "          9.4314e-02,  1.3531e-01, -8.8651e-01, -5.7966e-01, -3.3632e-01,\n",
       "          3.1987e-01, -1.9380e-01,  2.8006e-01,  5.9293e-01,  5.4405e-01,\n",
       "          3.7746e-01,  1.5074e-01,  1.3050e-01,  1.8917e-01, -8.2223e-03,\n",
       "         -3.1581e-01,  3.0006e-01,  2.2615e-01, -4.7198e-01,  2.9255e-01,\n",
       "          3.9098e-01,  7.1997e-01, -6.5656e-01, -5.7877e-02, -7.4665e-01,\n",
       "          4.5998e-01, -1.7215e-01,  3.4801e-01, -3.4560e-01,  3.4529e-01,\n",
       "          4.9162e-01,  3.4105e-01,  5.2297e-01, -5.5147e-01,  2.9069e-02,\n",
       "         -1.3466e+00,  1.6773e-01, -2.8781e-01, -6.5206e-01, -2.8571e-01,\n",
       "          4.4820e-01, -8.2600e-01,  2.7299e-01, -1.6052e-01, -4.9491e-01,\n",
       "         -4.4123e-01, -3.0832e-02,  3.5858e-01, -3.2000e-01, -3.6482e-01,\n",
       "         -4.6250e-01,  6.8682e-01,  6.2212e-01, -6.7616e-01,  4.4652e-01,\n",
       "          6.3600e-02,  5.6310e-01,  9.1847e-01,  4.4499e-01,  4.0972e-01,\n",
       "          6.3158e-01, -4.8508e-01, -4.0152e-01, -5.0366e-01, -4.5478e-02,\n",
       "         -4.5590e-01, -7.5448e-01,  6.8782e-01, -3.8171e-01,  2.5105e-01,\n",
       "          1.3606e-01,  1.9530e-01, -1.6731e-01,  1.8509e-01,  2.4151e-01,\n",
       "         -1.2497e-01,  7.3939e-02,  8.0915e-02,  4.0503e-01, -4.4470e-01,\n",
       "          1.4211e-01, -6.2766e-02, -1.0847e+00,  1.5061e-02,  3.1442e-01,\n",
       "          6.1792e-02, -2.7084e-01, -9.5111e-02,  8.0670e-01,  3.4658e-01,\n",
       "          1.7020e-01, -3.4658e-01,  6.2307e-01, -3.1217e-01,  6.7439e-02,\n",
       "         -4.2302e-01,  6.6040e-01,  2.8155e-01, -3.0294e-02,  1.3720e-01,\n",
       "         -4.0739e-01,  4.6180e-01,  7.8493e-01, -2.0656e-01, -6.3212e-02,\n",
       "         -8.2126e-01,  1.9584e-01, -8.8633e-02, -1.0430e+00,  2.6545e-01,\n",
       "          1.2379e+00,  6.7183e-01, -6.8933e-01,  2.3559e-01,  6.4715e-01,\n",
       "          1.2310e-01, -2.8892e-01,  2.7236e-01, -4.1748e-01,  9.3375e-01,\n",
       "          1.2763e-01, -4.8758e-01,  4.1386e-01, -5.7510e-01,  3.0247e-02,\n",
       "         -5.2484e-01, -3.5384e-01, -4.3309e-01, -5.0379e-01, -1.4411e-01,\n",
       "         -9.1641e-02,  1.3801e-01, -2.2796e-01,  4.0613e-01, -5.3039e-01,\n",
       "          2.8264e-01,  1.4536e-01, -3.8260e-01, -1.0636e-01, -1.9395e-01,\n",
       "          2.4888e-01,  3.9480e-01, -1.9706e-01, -9.8293e-02, -5.3219e-01,\n",
       "          3.9783e-01, -3.8129e-01,  1.8971e-01, -3.3756e-01, -6.2739e-01,\n",
       "         -7.0887e-01,  3.7720e-01,  2.1101e-01, -5.8501e-01, -4.8129e-01,\n",
       "          2.2947e-01, -1.3340e-01,  1.1180e+00,  6.6438e-02,  4.8484e-01,\n",
       "         -2.2995e-01, -3.2569e-01,  4.2933e-02,  3.4589e-01, -2.4830e-01,\n",
       "          1.0948e-01, -1.3069e-01, -5.7944e-01, -1.9196e-01, -7.8060e-02,\n",
       "          1.1934e-01, -2.3422e-01, -2.1563e-01, -3.3932e-01,  8.7396e-02,\n",
       "          2.8107e-01, -4.7970e-01,  2.1479e-01, -1.1446e-01, -5.0093e-01,\n",
       "         -1.2180e-01,  5.6778e-01, -1.7505e-01,  5.0499e-01, -6.7217e-01,\n",
       "          3.6737e-01, -1.8358e-01,  2.3660e-01,  2.9736e-02, -3.5624e-01,\n",
       "         -1.1451e-01,  6.5315e-01, -4.0913e-01, -5.6405e-01,  7.2596e-01,\n",
       "         -2.3398e-01,  1.7279e-01, -4.2138e-01,  1.0075e-01,  1.4012e-01,\n",
       "          1.6682e-01,  2.0844e-01,  7.5970e-02,  2.4352e-01,  1.3455e-01,\n",
       "          1.4141e-01,  1.7959e-01,  1.4143e+00,  1.2107e-01,  3.6410e-01,\n",
       "         -5.9567e-01,  4.5288e-02,  4.0947e-02, -6.3150e-03, -1.5476e-02,\n",
       "         -5.4967e-02,  4.0418e-02,  5.3073e-01,  3.6719e-01,  5.9932e-03,\n",
       "         -6.0457e-01, -4.0461e-02,  3.0526e-01,  2.2857e-01,  3.1699e-01,\n",
       "         -6.1115e-01, -3.5432e-01, -1.7680e-01,  3.0425e-01,  4.4631e-01,\n",
       "         -8.0750e-02,  3.3443e-02, -1.7067e-01,  2.9058e-01,  1.1439e-01,\n",
       "         -1.2962e-02,  2.6377e-01, -2.9394e-01, -2.3483e-01,  2.1922e-01,\n",
       "          7.9876e-01,  9.3168e-01, -1.6078e-01, -6.8043e-02, -2.8835e-01,\n",
       "         -3.7442e-02,  8.7288e-01, -3.0851e-02, -2.6066e-01,  8.7559e-01,\n",
       "          5.2805e-02,  4.2036e-01, -4.4657e-01, -6.0128e-01,  2.4042e-01,\n",
       "          1.0409e+00,  4.5662e-02, -5.9405e-01, -7.7239e-02,  4.4467e-01,\n",
       "          2.8607e-01,  3.1795e-01,  4.0720e-01, -4.4101e-01,  1.0156e-01,\n",
       "          8.4664e-02,  2.2385e-01,  3.2971e-02,  8.5590e-01, -8.3133e-02,\n",
       "          2.4421e-01, -2.5320e-01,  3.0961e-01,  3.2430e-01,  7.6288e-01,\n",
       "         -4.1389e-01, -7.4261e-01, -1.0888e-01,  3.0744e-02, -8.4961e-02,\n",
       "         -3.3006e-01, -9.4410e-02, -1.7684e-01,  9.1215e-02,  3.9247e-01,\n",
       "         -1.6165e-01, -2.5071e-01,  3.2119e-01, -5.8943e-01,  5.3371e-01,\n",
       "         -3.9494e-01,  4.5296e-01, -3.2192e-02,  2.9046e-02,  9.0094e-02,\n",
       "          8.0394e-02, -3.8424e-01, -2.4358e-01, -5.8893e-01, -1.6014e-01,\n",
       "         -5.0848e-01, -2.1314e-02, -5.0818e-01, -9.5969e-02,  4.0968e-01,\n",
       "          3.0001e-01, -5.3867e-01,  2.3648e-01, -5.9909e-01,  1.8138e-01,\n",
       "          5.6676e-01,  3.3343e-01,  3.6125e-02, -7.1760e-01, -2.8207e-01,\n",
       "         -1.7438e-01, -9.5262e-02,  2.1486e-01, -5.2020e-02, -1.4042e-04,\n",
       "         -1.5751e-01, -6.5503e-02,  3.1234e-01, -1.2235e-01, -1.2310e-01,\n",
       "         -2.6430e-01,  6.9013e-01, -8.8553e-01,  1.4780e-01, -4.3434e-01,\n",
       "          9.2152e-03, -5.1354e-01, -4.0582e-01,  4.0951e-02, -9.0218e-01,\n",
       "          1.7678e-01,  8.2160e-01,  9.0611e-02,  1.0791e-01, -5.0325e-01,\n",
       "          5.2987e-01, -4.0699e-01,  1.9170e-01, -2.1431e-01,  4.5255e-01,\n",
       "          5.6901e-01,  2.7348e-01, -4.4278e-01,  1.8978e-01,  1.3639e-03,\n",
       "          2.6983e-01,  6.8033e-01, -4.7854e-01,  8.3501e-01,  5.7284e-01,\n",
       "          5.5622e-01, -3.8540e-01, -1.7700e-02,  3.0426e-01,  2.9132e-01,\n",
       "         -2.5081e-01, -5.4555e-01,  4.0296e-02, -3.4137e-01,  4.2379e-01,\n",
       "         -5.2939e-01,  1.7819e-01, -1.2489e-02,  5.5839e-01,  1.0675e-01,\n",
       "          3.1746e-01, -2.8264e-01, -2.1325e-01], device='cuda:0'))"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bert_watch1 = embed1[31]\n",
    "bert_watch2 = embed2[13]\n",
    "bert_watch1, bert_watch2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "qvTJj1vcPIpg",
    "outputId": "7e45ec4b-d3c3-4be0-f686-79c05e2930cf"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(0.3504, device='cuda:0')"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similarity = nn.CosineSimilarity(dim=0, eps=1e-6)\n",
    "similarity(bert_watch1, bert_watch2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qt9Wl8aIPIpg"
   },
   "source": [
    "## Document Embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "id": "JgU-haTVPIpg"
   },
   "outputs": [],
   "source": [
    "documents = [Sentence(watch1), Sentence(watch2)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "eLbRmOR_PIpg",
    "outputId": "46005278-fb47-48a4-d0d7-a5a686d04cd9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Sentence[58]: \" The Hatter was the first to break the silence. `What day of the month is it?' he said, turning to Alice:  he had taken his watch out of his pocket, and was looking at it uneasily, shaking it every now and then, and holding it to his ear.\",\n",
       " Sentence[48]: \" Alice thought this a very curious thing, and she went nearer to watch them, and just as she came up to them she heard one of them say, `Look out now, Five!  Don't go splashing paint over me like that!\"]"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from flair.embeddings import TransformerDocumentEmbeddings\n",
    "bert_doc = TransformerDocumentEmbeddings('bert-base-uncased')\n",
    "bert_doc.embed(documents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UB6GBlxYPIpg",
    "outputId": "78fd2791-23c2-4442-99bd-a927be2d4224"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-6.4245e-02,  3.5365e-01, -2.4962e-01, -5.3912e-01, -1.9917e-01,\n",
       "        -2.7712e-01,  1.6942e-01,  1.0867e-01, -4.0799e-01, -1.4945e-01,\n",
       "         4.4678e-02, -1.9687e-01, -3.0574e-01,  9.6215e-02,  2.8053e-01,\n",
       "         9.0462e-01, -3.0250e-01,  1.7854e-01, -1.6856e-01, -1.7227e-01,\n",
       "        -1.8270e-01,  4.5121e-01,  3.6129e-01,  1.2309e-01, -1.2047e-02,\n",
       "        -5.0255e-01,  4.2645e-01, -3.7184e-02, -1.7052e-01, -4.8920e-02,\n",
       "         1.0503e-01,  3.3456e-01, -3.8483e-02, -4.7287e-01, -7.4431e-02,\n",
       "         2.3990e-01, -8.3232e-02,  1.7974e-01,  4.8939e-01,  9.3026e-03,\n",
       "        -4.2722e-01,  2.6337e-01, -1.3217e-01, -1.3950e-01, -1.1859e-01,\n",
       "         4.5835e-02, -3.9031e+00, -1.0726e-01,  5.1103e-02, -9.9710e-02,\n",
       "         1.5913e-01, -9.5331e-02,  2.1513e-01,  4.7707e-01,  2.1226e-01,\n",
       "         4.3450e-01, -4.5154e-01,  1.8344e-01, -1.0171e-01, -3.6373e-01,\n",
       "         8.9560e-01,  3.5931e-01, -2.8519e-01,  2.9427e-02,  3.3968e-01,\n",
       "         4.1062e-01, -2.1051e-02,  2.7801e-02,  2.8888e-01,  4.5761e-01,\n",
       "        -3.8254e-01, -1.9211e-01, -2.6551e-01,  1.7064e-01, -2.0920e-01,\n",
       "        -3.3608e-01, -1.0044e-01, -2.9611e-01, -5.6573e-01,  4.3841e-01,\n",
       "        -8.9239e-02, -1.4645e-02,  2.2009e-01,  2.5755e-01, -2.6711e-01,\n",
       "        -1.4440e-01, -2.8944e-01,  2.2547e-01,  6.0303e-01,  1.0250e-01,\n",
       "        -2.0606e-01, -1.5392e-01, -3.2987e-03,  7.7605e-02,  1.8428e-01,\n",
       "         1.6114e-01,  2.5668e-01, -8.3616e-02,  2.7078e-01, -5.4946e-02,\n",
       "         4.9633e-01, -2.7734e-01,  4.1486e-01, -5.3346e-01, -7.8131e-01,\n",
       "         1.7047e-01,  2.8687e-01,  4.9673e-02,  4.6922e-01, -1.6500e+00,\n",
       "         4.6991e-01, -1.5328e-01,  2.7729e-01, -1.0157e+00,  1.5022e-01,\n",
       "        -1.7466e-01,  4.1232e-02, -4.4218e-01, -5.0299e-01,  1.1937e-01,\n",
       "         1.8505e-01,  1.7673e-01,  2.1935e-01, -1.3052e+00, -3.4139e-01,\n",
       "        -3.8845e-02, -2.4191e-01,  2.5905e-01,  1.2551e-01, -1.7338e-02,\n",
       "         4.4672e-01,  3.3219e-01, -2.1076e-01, -2.6042e-01,  2.1485e-01,\n",
       "         1.6804e-01, -1.6398e-01,  1.1147e-01, -1.4731e-01,  1.0370e-01,\n",
       "        -2.0025e-01, -1.7985e-01, -2.5822e+00, -9.3411e-02,  5.8352e-01,\n",
       "         4.2851e-01,  1.3719e-01, -4.4423e-01, -2.4915e-03,  6.3892e-01,\n",
       "         4.1029e-01, -3.2187e-01,  5.8879e-02, -2.1344e-01, -4.0472e-02,\n",
       "        -4.0414e-01,  3.8056e-01,  1.7888e-01, -4.1893e-01,  1.6880e-01,\n",
       "        -4.6562e-02, -3.3560e-01,  1.3598e-02,  2.4541e-01, -1.9910e-01,\n",
       "         5.3978e-01,  5.7400e-01,  5.7995e-01, -3.8400e-02, -4.4134e-02,\n",
       "         3.8312e-01, -1.6357e-02,  7.0812e-01, -1.6653e-01, -4.0108e-01,\n",
       "         8.2582e-02,  4.2481e-01, -1.2611e-01,  4.2772e-01, -7.9697e-02,\n",
       "        -7.4779e-01,  7.4372e-02,  2.0130e-01,  8.0436e-01,  8.2707e-01,\n",
       "         3.8594e-01, -2.6690e-02, -2.8838e-01,  2.1367e-01,  2.3883e-01,\n",
       "         1.4772e-01, -1.7737e-01, -1.6458e-01, -4.7632e-01, -9.9866e-02,\n",
       "        -7.7609e-02,  3.6381e-01, -5.1024e-01, -3.6723e-01, -2.4667e-01,\n",
       "        -2.0106e-01,  1.8970e-01,  1.2936e-01, -7.0922e-01,  1.8065e-01,\n",
       "         4.2441e+00,  3.2519e-01, -3.5622e-01,  7.9891e-02,  4.0457e-01,\n",
       "        -5.2027e-01, -4.0209e-01,  1.1543e-01,  5.4013e-01, -8.5186e-01,\n",
       "         5.7481e-01,  3.0116e-01,  3.0603e-01,  3.0003e-02, -3.7785e-01,\n",
       "        -2.3895e-01,  1.0855e-01, -7.3348e-02, -2.2568e-02, -1.8636e-01,\n",
       "         5.5137e-01, -2.5551e-01,  5.6108e-01,  8.9961e-02, -1.4174e+00,\n",
       "         1.9548e-01,  1.8128e-01,  5.7270e-01,  3.7286e-01,  2.1192e-02,\n",
       "         2.0576e-01, -2.7981e-01, -4.0403e-02, -5.4554e-02, -2.4785e-01,\n",
       "        -2.4466e-01, -4.0418e-03,  8.9291e-02,  5.5506e-01, -1.7438e-01,\n",
       "         4.4969e-01,  3.9400e-01, -5.3704e-01,  6.6374e-01,  3.8592e-01,\n",
       "         6.9357e-01,  6.0498e-01, -2.1726e-01, -1.0124e-01, -2.2627e-01,\n",
       "         5.7905e-01,  4.2297e-01, -1.2389e-01,  2.0031e-01,  4.5129e-01,\n",
       "         1.7262e-01,  1.0963e-02,  3.6436e-01,  5.7858e-01, -5.3210e-01,\n",
       "        -1.4753e-02,  6.1678e-01, -3.0254e-01, -3.7331e-01,  4.2933e-02,\n",
       "        -2.8411e-01,  2.5481e-01, -3.1135e-01, -2.4889e+00,  1.2208e-01,\n",
       "        -8.3915e-02, -1.0063e-01,  1.6028e-01,  9.8308e-02, -1.5816e-01,\n",
       "        -3.4995e-02, -1.4911e-01, -8.0463e-01,  5.4030e-01,  7.1562e-02,\n",
       "         1.7398e-01,  1.4491e-01, -7.0773e-01,  3.9191e-01,  1.0117e-01,\n",
       "        -2.8370e-01,  2.8936e-01,  1.5133e-01,  5.0763e-02, -2.4023e-01,\n",
       "         4.4110e-01, -3.9492e-02,  2.6435e-01,  3.1356e-01, -1.8297e-01,\n",
       "        -2.4273e-01, -1.7542e-01,  1.4501e-01,  3.0055e-01, -2.8845e-01,\n",
       "        -1.5901e-01,  1.7493e-01, -2.9559e-01, -3.4490e+00,  1.3099e-01,\n",
       "        -3.8233e-01, -1.0336e-01,  6.8291e-01,  6.9417e-01,  6.1574e-01,\n",
       "        -8.2423e-02, -1.2499e-01, -1.2243e-01,  2.8974e-01, -4.1564e-01,\n",
       "         6.8800e-02,  6.7222e-01,  5.1716e-01, -3.0253e-02,  3.6390e-01,\n",
       "        -5.9201e-01,  2.2436e-01, -2.7201e-01, -8.8383e-02, -5.5831e-01,\n",
       "         1.0460e-01, -3.3686e-01,  4.6233e-01,  6.3347e-01, -5.7720e-01,\n",
       "        -4.1045e-01, -6.3033e-02, -2.6181e-01,  2.5220e-01, -5.7010e-01,\n",
       "        -4.4242e-02, -2.5707e-03, -2.6010e-01, -2.0902e-01,  2.1671e-01,\n",
       "        -1.5338e-01,  2.3135e-02,  8.3759e-02,  2.4785e-01,  8.1829e-01,\n",
       "         4.6562e-01,  1.6161e-01,  1.5305e-01, -2.2000e-01, -2.5913e-01,\n",
       "        -3.0355e-01,  2.0841e-01, -8.6140e-02, -1.0182e-01,  7.8481e-01,\n",
       "         1.0315e+00, -5.8092e-01, -2.5592e-01,  1.4235e-01,  2.2627e-01,\n",
       "        -3.8135e-01, -1.7177e-01,  6.8986e-01,  9.5218e-01, -4.2518e-01,\n",
       "         2.9856e-01, -7.7871e-01,  4.6342e-01,  3.8082e-01,  3.3461e-01,\n",
       "        -3.6359e-01,  1.3449e-01,  6.0971e-02, -6.3682e-01,  6.5791e-01,\n",
       "        -3.4900e-01, -1.6919e+00, -1.1290e-01, -8.9324e-03,  7.0164e-01,\n",
       "        -2.4298e-01, -5.0133e-01, -9.4679e-02,  9.2998e-02, -6.9514e-01,\n",
       "         1.6732e-01,  1.0738e-01, -3.5135e-01, -4.1886e-01, -5.6835e-02,\n",
       "         1.8975e-01, -6.7744e-01, -5.0675e-02, -2.8237e-01, -9.4651e-02,\n",
       "         2.3333e-01,  2.1024e-01, -1.3943e-01,  4.7849e-01, -3.5598e-02,\n",
       "        -5.0288e-02,  2.2644e-01,  1.7870e-01, -8.6450e-02, -4.6139e-01,\n",
       "        -9.7895e-02,  9.5692e-03, -6.0332e-01, -1.0339e-01, -6.9934e-01,\n",
       "         8.6564e-01,  2.4269e-01,  1.2407e-01, -4.4824e-01,  6.2005e-02,\n",
       "         1.1447e-01, -4.3741e-02,  1.5308e+00, -4.7716e-01,  4.9675e-01,\n",
       "         8.2663e-01,  4.4054e-01, -8.6615e-02,  2.1151e-01, -1.6980e-02,\n",
       "         3.6449e-02,  5.2057e-02, -4.3879e-01, -2.0526e-01, -1.0826e-01,\n",
       "        -2.0654e-01,  5.4324e-02, -2.3006e-01, -2.3910e-01,  3.5886e-01,\n",
       "        -3.6865e-01, -4.1345e-01,  3.2852e-01, -2.1309e-01, -7.7967e-01,\n",
       "        -4.0635e-01,  4.1312e-01,  4.5644e-01,  5.6297e-01, -4.0113e-02,\n",
       "        -7.8324e-02,  3.5196e-01, -4.0508e-01,  4.9119e-01,  2.8871e-01,\n",
       "         1.2004e-02,  2.1122e-02,  5.8201e-01,  7.2554e-02, -1.2406e-01,\n",
       "        -2.2747e-01, -3.6875e-01,  1.2822e-01, -1.3219e-02, -2.5432e-01,\n",
       "        -5.4382e-01, -1.6104e-01, -4.9271e-01, -2.7153e-05, -1.6866e-01,\n",
       "        -2.0367e+00, -1.8150e-01,  7.2432e-01,  2.6644e-01,  2.3441e-01,\n",
       "         1.3906e-01, -7.0422e-03,  4.3320e-01, -3.9089e-01,  3.8913e-01,\n",
       "        -1.3505e-01,  1.8608e-01, -2.7714e-01,  7.7001e-02, -3.5178e-01,\n",
       "         3.1395e-01, -6.1126e-01, -1.9384e-01,  3.4077e-02,  7.6535e-02,\n",
       "        -6.3849e-01,  9.1558e-02,  6.1446e-01, -9.3747e-02,  2.7309e-02,\n",
       "        -2.9999e-01,  8.1415e-02,  2.3125e-01,  4.6290e-02,  1.2134e-01,\n",
       "         1.3592e-01, -5.6097e-01, -2.1794e-01, -6.8162e-01,  6.4019e-01,\n",
       "        -1.8568e-01, -1.6626e-01,  3.4131e-01,  1.8143e-01,  1.4511e-01,\n",
       "         4.1716e-02,  3.5290e-01,  2.6542e-01, -3.3044e-01,  6.5737e-01,\n",
       "        -4.8462e-01, -4.2369e-01,  5.4556e-01, -4.1632e-01, -7.4869e-02,\n",
       "        -2.7194e-01, -5.6729e-02, -1.9846e-01, -2.5328e-01, -8.4373e-01,\n",
       "        -2.5077e-01, -7.6245e-02,  1.3954e-01, -7.3810e-01, -7.1694e-02,\n",
       "         3.7180e-01, -6.6641e-01, -7.3948e-02,  1.0740e-02,  6.6874e-02,\n",
       "        -5.4790e-01, -2.3494e-01, -5.4339e-02, -2.7594e-01,  2.3641e-01,\n",
       "         9.0829e-02, -2.4923e-02, -6.5618e-01, -1.5407e-01,  6.5148e-03,\n",
       "        -1.8896e-01, -4.8536e-01,  4.1967e-01,  4.9990e-01, -2.3470e-01,\n",
       "         2.3999e-01,  2.2489e-02,  7.1899e-02,  3.7770e-01,  2.2666e-01,\n",
       "        -4.6463e-02,  7.9462e-02, -1.3600e-01,  2.6624e-01, -1.0557e-01,\n",
       "        -5.1542e-01, -2.1026e-01, -3.6529e-01,  3.7681e-01,  4.2558e-01,\n",
       "        -2.0674e-01,  1.7708e-01, -9.5000e-01,  3.5931e-01, -6.9485e-01,\n",
       "         2.6662e-01, -3.8393e-01, -1.4876e-01,  3.5606e-01, -1.9180e-01,\n",
       "         5.2490e-02,  6.9162e-01,  1.2743e-01, -2.9465e-01, -6.5748e-01,\n",
       "        -1.3319e-01, -1.6700e-01,  3.4043e-01,  1.0655e-01,  1.4711e-01,\n",
       "        -5.7975e-01, -1.9442e-01, -5.5820e-01,  1.5113e+00,  2.5854e-01,\n",
       "        -7.7182e-02,  1.9113e-01, -2.0155e-01,  5.2039e-02, -6.6809e-02,\n",
       "         7.7022e-01,  4.3338e-01,  2.1713e-01, -1.9559e-01,  1.1622e-01,\n",
       "         6.9701e-02,  1.9887e-01,  4.4560e-01, -2.0377e-01,  9.3891e-02,\n",
       "        -3.8858e-01, -1.0287e+00, -8.7257e-02,  3.0796e-01,  1.1579e-01,\n",
       "         2.2186e-01,  2.4396e-02, -2.2758e-01,  7.7906e-01,  8.9298e-02,\n",
       "        -5.7492e-01, -2.8990e-01,  5.5056e-01, -3.1180e-01,  5.1760e-01,\n",
       "        -2.1881e-01,  3.5772e-01, -9.8653e-01,  3.4400e-02, -1.5934e-01,\n",
       "        -2.6761e-01,  4.0133e-02,  7.4575e-02,  3.0935e-01,  2.3991e-02,\n",
       "        -2.4070e-01,  1.9046e-01, -3.6814e-01,  3.0127e-01,  1.0484e-01,\n",
       "         3.0938e-02,  3.2499e-01,  4.0934e-01, -1.7192e-01,  5.4189e-01,\n",
       "         3.5965e-01,  1.4977e-01, -2.0299e-01,  2.5858e-02, -2.3124e-01,\n",
       "        -5.2681e-01, -5.0653e-02, -4.0077e-01,  3.5185e-02,  6.1388e-01,\n",
       "         1.7713e-01, -5.8146e-01,  1.2276e-01,  2.9924e-02, -5.3960e-02,\n",
       "        -4.8978e-01,  8.5425e-02, -1.6308e-01,  2.2881e-02,  2.6261e-01,\n",
       "         2.0515e-01,  3.4557e-01,  2.7707e-01,  3.1900e-01,  6.0790e-01,\n",
       "        -5.5657e-01, -4.6023e-01, -1.5954e+00,  1.5275e-01,  3.5691e-01,\n",
       "         4.9637e-01, -1.7432e-01,  1.8391e-01,  5.2575e-01,  1.6172e-01,\n",
       "        -1.6158e-01,  1.2285e-01,  1.7329e-01,  1.0693e+00,  4.1782e-01,\n",
       "         2.6315e-01, -2.6413e-02, -3.4535e-01,  3.1312e-01,  3.1309e-03,\n",
       "        -1.3553e-01,  9.2502e-02,  1.8213e-01,  1.0432e-01, -5.5065e-02,\n",
       "        -5.2151e-01, -5.9766e-01, -3.0186e-01,  1.1254e-02,  6.2411e-01,\n",
       "        -2.4277e-01,  2.0318e-01, -1.6011e-01,  8.6397e-01, -2.2691e-02,\n",
       "        -4.4046e-01, -6.2857e-02, -3.9069e-01,  1.0784e-01,  1.1981e-01,\n",
       "         4.9630e-01, -3.8379e-03, -4.2250e-01,  1.0451e+00, -2.6349e-01,\n",
       "        -1.4386e-01,  2.8545e-01,  2.6442e-01,  5.0848e-01, -3.4545e-01,\n",
       "         2.0938e-01,  3.9765e-01, -2.8668e-01, -2.9289e-01,  1.0622e-01,\n",
       "         1.2800e-01,  4.2874e-01, -7.5396e-02,  2.4798e-01,  8.1016e-02,\n",
       "        -2.1906e-02, -2.7718e-01, -7.3759e-02, -5.9759e-02, -1.1127e-01,\n",
       "        -1.2804e-01,  2.8441e-01, -1.5104e-01, -2.0100e-01,  7.6934e-01,\n",
       "         3.0968e-01, -5.9233e-01, -3.4167e-01, -1.5913e-01, -5.3265e-02,\n",
       "         2.1583e-01, -3.5234e-01, -1.2197e-01,  3.0294e-01,  7.3882e-01,\n",
       "         1.3297e-01, -3.4018e-01, -8.1062e-02, -2.5973e-01,  1.7507e-01,\n",
       "         2.3974e-02,  4.1746e-01, -6.2069e+00,  1.1619e-01, -5.7523e-01,\n",
       "        -2.6148e-01, -3.7661e-01, -2.8330e-01,  1.3297e-01,  3.6957e-01,\n",
       "         7.4661e-02, -3.4777e-01,  1.5740e-01,  3.4407e-01, -5.0272e-01,\n",
       "         1.7432e-01,  7.9398e-01,  7.3562e-01], device='cuda:0')"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "documents[0].embedding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "zX4ZkKaPPIph",
    "outputId": "1b390558-d731-4ca9-fe11-589082919587"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([], device='cuda:0')"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "documents[0].tokens[31].embedding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "id": "w8XT_0ahPIph"
   },
   "outputs": [],
   "source": [
    "def get_embeddings(embeddings, sentence):\n",
    "    sent = Sentence(sentence)\n",
    "    embeddings.embed(sent)\n",
    "    if len(sent.embedding):\n",
    "        return sent.embedding.float()\n",
    "    else:\n",
    "        return torch.stack([token.embedding for token in sent.tokens]).float()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2yt6esWrPIph",
    "outputId": "de9d7620-f957-40d4-a660-50aa995f7f91"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-6.4245e-02,  3.5365e-01, -2.4962e-01, -5.3912e-01, -1.9917e-01,\n",
       "        -2.7712e-01,  1.6942e-01,  1.0867e-01, -4.0799e-01, -1.4945e-01,\n",
       "         4.4679e-02, -1.9687e-01, -3.0574e-01,  9.6215e-02,  2.8053e-01,\n",
       "         9.0462e-01, -3.0250e-01,  1.7854e-01, -1.6856e-01, -1.7227e-01,\n",
       "        -1.8270e-01,  4.5121e-01,  3.6128e-01,  1.2309e-01, -1.2047e-02,\n",
       "        -5.0255e-01,  4.2645e-01, -3.7184e-02, -1.7052e-01, -4.8920e-02,\n",
       "         1.0503e-01,  3.3456e-01, -3.8483e-02, -4.7287e-01, -7.4431e-02,\n",
       "         2.3990e-01, -8.3232e-02,  1.7974e-01,  4.8939e-01,  9.3027e-03,\n",
       "        -4.2722e-01,  2.6337e-01, -1.3217e-01, -1.3950e-01, -1.1859e-01,\n",
       "         4.5834e-02, -3.9031e+00, -1.0726e-01,  5.1103e-02, -9.9710e-02,\n",
       "         1.5913e-01, -9.5331e-02,  2.1513e-01,  4.7707e-01,  2.1226e-01,\n",
       "         4.3450e-01, -4.5154e-01,  1.8344e-01, -1.0171e-01, -3.6373e-01,\n",
       "         8.9560e-01,  3.5931e-01, -2.8519e-01,  2.9427e-02,  3.3968e-01,\n",
       "         4.1062e-01, -2.1051e-02,  2.7801e-02,  2.8888e-01,  4.5761e-01,\n",
       "        -3.8254e-01, -1.9211e-01, -2.6551e-01,  1.7064e-01, -2.0920e-01,\n",
       "        -3.3608e-01, -1.0044e-01, -2.9611e-01, -5.6573e-01,  4.3841e-01,\n",
       "        -8.9239e-02, -1.4645e-02,  2.2009e-01,  2.5755e-01, -2.6711e-01,\n",
       "        -1.4440e-01, -2.8944e-01,  2.2546e-01,  6.0303e-01,  1.0250e-01,\n",
       "        -2.0606e-01, -1.5392e-01, -3.2986e-03,  7.7604e-02,  1.8428e-01,\n",
       "         1.6114e-01,  2.5668e-01, -8.3616e-02,  2.7078e-01, -5.4946e-02,\n",
       "         4.9633e-01, -2.7734e-01,  4.1486e-01, -5.3346e-01, -7.8131e-01,\n",
       "         1.7047e-01,  2.8687e-01,  4.9674e-02,  4.6922e-01, -1.6500e+00,\n",
       "         4.6991e-01, -1.5328e-01,  2.7729e-01, -1.0157e+00,  1.5022e-01,\n",
       "        -1.7466e-01,  4.1232e-02, -4.4218e-01, -5.0299e-01,  1.1937e-01,\n",
       "         1.8505e-01,  1.7673e-01,  2.1935e-01, -1.3052e+00, -3.4139e-01,\n",
       "        -3.8845e-02, -2.4191e-01,  2.5906e-01,  1.2551e-01, -1.7338e-02,\n",
       "         4.4672e-01,  3.3219e-01, -2.1076e-01, -2.6042e-01,  2.1485e-01,\n",
       "         1.6804e-01, -1.6398e-01,  1.1147e-01, -1.4731e-01,  1.0370e-01,\n",
       "        -2.0025e-01, -1.7985e-01, -2.5822e+00, -9.3411e-02,  5.8352e-01,\n",
       "         4.2851e-01,  1.3719e-01, -4.4423e-01, -2.4913e-03,  6.3892e-01,\n",
       "         4.1029e-01, -3.2187e-01,  5.8879e-02, -2.1344e-01, -4.0472e-02,\n",
       "        -4.0414e-01,  3.8056e-01,  1.7888e-01, -4.1893e-01,  1.6880e-01,\n",
       "        -4.6562e-02, -3.3560e-01,  1.3598e-02,  2.4541e-01, -1.9910e-01,\n",
       "         5.3978e-01,  5.7400e-01,  5.7995e-01, -3.8400e-02, -4.4135e-02,\n",
       "         3.8312e-01, -1.6357e-02,  7.0812e-01, -1.6653e-01, -4.0108e-01,\n",
       "         8.2581e-02,  4.2481e-01, -1.2611e-01,  4.2772e-01, -7.9696e-02,\n",
       "        -7.4779e-01,  7.4372e-02,  2.0130e-01,  8.0436e-01,  8.2707e-01,\n",
       "         3.8594e-01, -2.6690e-02, -2.8838e-01,  2.1367e-01,  2.3883e-01,\n",
       "         1.4773e-01, -1.7737e-01, -1.6458e-01, -4.7632e-01, -9.9866e-02,\n",
       "        -7.7609e-02,  3.6381e-01, -5.1024e-01, -3.6723e-01, -2.4667e-01,\n",
       "        -2.0106e-01,  1.8970e-01,  1.2936e-01, -7.0922e-01,  1.8065e-01,\n",
       "         4.2441e+00,  3.2519e-01, -3.5622e-01,  7.9890e-02,  4.0457e-01,\n",
       "        -5.2027e-01, -4.0209e-01,  1.1543e-01,  5.4013e-01, -8.5187e-01,\n",
       "         5.7481e-01,  3.0116e-01,  3.0603e-01,  3.0002e-02, -3.7785e-01,\n",
       "        -2.3895e-01,  1.0855e-01, -7.3348e-02, -2.2569e-02, -1.8636e-01,\n",
       "         5.5137e-01, -2.5551e-01,  5.6109e-01,  8.9961e-02, -1.4174e+00,\n",
       "         1.9548e-01,  1.8128e-01,  5.7270e-01,  3.7286e-01,  2.1192e-02,\n",
       "         2.0576e-01, -2.7981e-01, -4.0403e-02, -5.4554e-02, -2.4785e-01,\n",
       "        -2.4466e-01, -4.0415e-03,  8.9292e-02,  5.5506e-01, -1.7438e-01,\n",
       "         4.4969e-01,  3.9400e-01, -5.3704e-01,  6.6374e-01,  3.8592e-01,\n",
       "         6.9357e-01,  6.0498e-01, -2.1726e-01, -1.0124e-01, -2.2627e-01,\n",
       "         5.7905e-01,  4.2297e-01, -1.2389e-01,  2.0031e-01,  4.5129e-01,\n",
       "         1.7262e-01,  1.0963e-02,  3.6436e-01,  5.7858e-01, -5.3210e-01,\n",
       "        -1.4754e-02,  6.1678e-01, -3.0254e-01, -3.7331e-01,  4.2933e-02,\n",
       "        -2.8411e-01,  2.5481e-01, -3.1135e-01, -2.4889e+00,  1.2208e-01,\n",
       "        -8.3916e-02, -1.0063e-01,  1.6028e-01,  9.8307e-02, -1.5816e-01,\n",
       "        -3.4994e-02, -1.4911e-01, -8.0463e-01,  5.4030e-01,  7.1562e-02,\n",
       "         1.7398e-01,  1.4491e-01, -7.0773e-01,  3.9191e-01,  1.0117e-01,\n",
       "        -2.8370e-01,  2.8936e-01,  1.5133e-01,  5.0763e-02, -2.4023e-01,\n",
       "         4.4110e-01, -3.9492e-02,  2.6435e-01,  3.1356e-01, -1.8297e-01,\n",
       "        -2.4273e-01, -1.7542e-01,  1.4501e-01,  3.0055e-01, -2.8845e-01,\n",
       "        -1.5901e-01,  1.7493e-01, -2.9559e-01, -3.4490e+00,  1.3099e-01,\n",
       "        -3.8233e-01, -1.0336e-01,  6.8291e-01,  6.9417e-01,  6.1574e-01,\n",
       "        -8.2423e-02, -1.2499e-01, -1.2243e-01,  2.8974e-01, -4.1564e-01,\n",
       "         6.8800e-02,  6.7222e-01,  5.1716e-01, -3.0253e-02,  3.6390e-01,\n",
       "        -5.9201e-01,  2.2436e-01, -2.7201e-01, -8.8383e-02, -5.5831e-01,\n",
       "         1.0460e-01, -3.3686e-01,  4.6233e-01,  6.3347e-01, -5.7720e-01,\n",
       "        -4.1045e-01, -6.3033e-02, -2.6181e-01,  2.5220e-01, -5.7010e-01,\n",
       "        -4.4242e-02, -2.5706e-03, -2.6010e-01, -2.0902e-01,  2.1671e-01,\n",
       "        -1.5338e-01,  2.3135e-02,  8.3758e-02,  2.4785e-01,  8.1829e-01,\n",
       "         4.6562e-01,  1.6161e-01,  1.5305e-01, -2.2000e-01, -2.5913e-01,\n",
       "        -3.0355e-01,  2.0841e-01, -8.6139e-02, -1.0182e-01,  7.8481e-01,\n",
       "         1.0315e+00, -5.8092e-01, -2.5592e-01,  1.4235e-01,  2.2627e-01,\n",
       "        -3.8135e-01, -1.7177e-01,  6.8986e-01,  9.5218e-01, -4.2518e-01,\n",
       "         2.9856e-01, -7.7871e-01,  4.6342e-01,  3.8082e-01,  3.3461e-01,\n",
       "        -3.6359e-01,  1.3449e-01,  6.0971e-02, -6.3682e-01,  6.5791e-01,\n",
       "        -3.4900e-01, -1.6919e+00, -1.1290e-01, -8.9325e-03,  7.0164e-01,\n",
       "        -2.4298e-01, -5.0133e-01, -9.4679e-02,  9.2998e-02, -6.9514e-01,\n",
       "         1.6732e-01,  1.0738e-01, -3.5135e-01, -4.1886e-01, -5.6835e-02,\n",
       "         1.8975e-01, -6.7744e-01, -5.0675e-02, -2.8237e-01, -9.4652e-02,\n",
       "         2.3333e-01,  2.1025e-01, -1.3943e-01,  4.7849e-01, -3.5598e-02,\n",
       "        -5.0288e-02,  2.2644e-01,  1.7870e-01, -8.6450e-02, -4.6139e-01,\n",
       "        -9.7895e-02,  9.5687e-03, -6.0332e-01, -1.0339e-01, -6.9934e-01,\n",
       "         8.6564e-01,  2.4269e-01,  1.2407e-01, -4.4824e-01,  6.2005e-02,\n",
       "         1.1447e-01, -4.3741e-02,  1.5308e+00, -4.7716e-01,  4.9675e-01,\n",
       "         8.2663e-01,  4.4054e-01, -8.6616e-02,  2.1151e-01, -1.6980e-02,\n",
       "         3.6449e-02,  5.2057e-02, -4.3879e-01, -2.0526e-01, -1.0826e-01,\n",
       "        -2.0654e-01,  5.4324e-02, -2.3006e-01, -2.3910e-01,  3.5886e-01,\n",
       "        -3.6865e-01, -4.1345e-01,  3.2852e-01, -2.1309e-01, -7.7967e-01,\n",
       "        -4.0635e-01,  4.1312e-01,  4.5644e-01,  5.6297e-01, -4.0113e-02,\n",
       "        -7.8324e-02,  3.5196e-01, -4.0508e-01,  4.9119e-01,  2.8871e-01,\n",
       "         1.2004e-02,  2.1122e-02,  5.8201e-01,  7.2554e-02, -1.2406e-01,\n",
       "        -2.2747e-01, -3.6875e-01,  1.2822e-01, -1.3220e-02, -2.5432e-01,\n",
       "        -5.4382e-01, -1.6104e-01, -4.9272e-01, -2.7074e-05, -1.6866e-01,\n",
       "        -2.0367e+00, -1.8150e-01,  7.2432e-01,  2.6644e-01,  2.3441e-01,\n",
       "         1.3906e-01, -7.0424e-03,  4.3320e-01, -3.9089e-01,  3.8913e-01,\n",
       "        -1.3505e-01,  1.8608e-01, -2.7714e-01,  7.7001e-02, -3.5178e-01,\n",
       "         3.1395e-01, -6.1126e-01, -1.9384e-01,  3.4077e-02,  7.6536e-02,\n",
       "        -6.3849e-01,  9.1557e-02,  6.1446e-01, -9.3747e-02,  2.7309e-02,\n",
       "        -2.9999e-01,  8.1415e-02,  2.3125e-01,  4.6289e-02,  1.2134e-01,\n",
       "         1.3592e-01, -5.6097e-01, -2.1794e-01, -6.8162e-01,  6.4019e-01,\n",
       "        -1.8568e-01, -1.6626e-01,  3.4131e-01,  1.8143e-01,  1.4511e-01,\n",
       "         4.1716e-02,  3.5290e-01,  2.6542e-01, -3.3044e-01,  6.5737e-01,\n",
       "        -4.8462e-01, -4.2369e-01,  5.4556e-01, -4.1632e-01, -7.4868e-02,\n",
       "        -2.7194e-01, -5.6729e-02, -1.9846e-01, -2.5328e-01, -8.4373e-01,\n",
       "        -2.5077e-01, -7.6245e-02,  1.3954e-01, -7.3810e-01, -7.1693e-02,\n",
       "         3.7180e-01, -6.6641e-01, -7.3948e-02,  1.0741e-02,  6.6875e-02,\n",
       "        -5.4790e-01, -2.3494e-01, -5.4339e-02, -2.7594e-01,  2.3641e-01,\n",
       "         9.0829e-02, -2.4924e-02, -6.5618e-01, -1.5407e-01,  6.5148e-03,\n",
       "        -1.8896e-01, -4.8536e-01,  4.1967e-01,  4.9990e-01, -2.3470e-01,\n",
       "         2.3999e-01,  2.2488e-02,  7.1898e-02,  3.7770e-01,  2.2666e-01,\n",
       "        -4.6463e-02,  7.9462e-02, -1.3600e-01,  2.6624e-01, -1.0557e-01,\n",
       "        -5.1542e-01, -2.1026e-01, -3.6529e-01,  3.7681e-01,  4.2558e-01,\n",
       "        -2.0673e-01,  1.7708e-01, -9.5000e-01,  3.5931e-01, -6.9485e-01,\n",
       "         2.6662e-01, -3.8393e-01, -1.4876e-01,  3.5606e-01, -1.9180e-01,\n",
       "         5.2491e-02,  6.9162e-01,  1.2743e-01, -2.9465e-01, -6.5748e-01,\n",
       "        -1.3319e-01, -1.6700e-01,  3.4043e-01,  1.0655e-01,  1.4711e-01,\n",
       "        -5.7975e-01, -1.9442e-01, -5.5820e-01,  1.5113e+00,  2.5854e-01,\n",
       "        -7.7182e-02,  1.9113e-01, -2.0155e-01,  5.2040e-02, -6.6809e-02,\n",
       "         7.7022e-01,  4.3337e-01,  2.1713e-01, -1.9559e-01,  1.1622e-01,\n",
       "         6.9701e-02,  1.9887e-01,  4.4560e-01, -2.0377e-01,  9.3891e-02,\n",
       "        -3.8858e-01, -1.0287e+00, -8.7257e-02,  3.0796e-01,  1.1579e-01,\n",
       "         2.2186e-01,  2.4396e-02, -2.2758e-01,  7.7906e-01,  8.9299e-02,\n",
       "        -5.7492e-01, -2.8990e-01,  5.5056e-01, -3.1180e-01,  5.1760e-01,\n",
       "        -2.1880e-01,  3.5772e-01, -9.8653e-01,  3.4400e-02, -1.5934e-01,\n",
       "        -2.6761e-01,  4.0133e-02,  7.4575e-02,  3.0935e-01,  2.3991e-02,\n",
       "        -2.4070e-01,  1.9046e-01, -3.6814e-01,  3.0126e-01,  1.0484e-01,\n",
       "         3.0939e-02,  3.2499e-01,  4.0934e-01, -1.7192e-01,  5.4189e-01,\n",
       "         3.5965e-01,  1.4977e-01, -2.0299e-01,  2.5858e-02, -2.3124e-01,\n",
       "        -5.2681e-01, -5.0653e-02, -4.0077e-01,  3.5185e-02,  6.1388e-01,\n",
       "         1.7713e-01, -5.8146e-01,  1.2276e-01,  2.9924e-02, -5.3960e-02,\n",
       "        -4.8978e-01,  8.5425e-02, -1.6308e-01,  2.2882e-02,  2.6261e-01,\n",
       "         2.0515e-01,  3.4557e-01,  2.7707e-01,  3.1900e-01,  6.0790e-01,\n",
       "        -5.5657e-01, -4.6023e-01, -1.5954e+00,  1.5275e-01,  3.5691e-01,\n",
       "         4.9637e-01, -1.7432e-01,  1.8391e-01,  5.2575e-01,  1.6172e-01,\n",
       "        -1.6158e-01,  1.2285e-01,  1.7329e-01,  1.0693e+00,  4.1782e-01,\n",
       "         2.6315e-01, -2.6412e-02, -3.4535e-01,  3.1312e-01,  3.1306e-03,\n",
       "        -1.3553e-01,  9.2502e-02,  1.8213e-01,  1.0432e-01, -5.5065e-02,\n",
       "        -5.2151e-01, -5.9766e-01, -3.0186e-01,  1.1254e-02,  6.2411e-01,\n",
       "        -2.4277e-01,  2.0318e-01, -1.6011e-01,  8.6397e-01, -2.2691e-02,\n",
       "        -4.4046e-01, -6.2856e-02, -3.9069e-01,  1.0784e-01,  1.1981e-01,\n",
       "         4.9630e-01, -3.8379e-03, -4.2250e-01,  1.0451e+00, -2.6349e-01,\n",
       "        -1.4386e-01,  2.8545e-01,  2.6442e-01,  5.0848e-01, -3.4544e-01,\n",
       "         2.0938e-01,  3.9765e-01, -2.8668e-01, -2.9289e-01,  1.0621e-01,\n",
       "         1.2800e-01,  4.2874e-01, -7.5396e-02,  2.4798e-01,  8.1016e-02,\n",
       "        -2.1905e-02, -2.7718e-01, -7.3759e-02, -5.9758e-02, -1.1127e-01,\n",
       "        -1.2804e-01,  2.8441e-01, -1.5104e-01, -2.0100e-01,  7.6934e-01,\n",
       "         3.0968e-01, -5.9233e-01, -3.4167e-01, -1.5913e-01, -5.3265e-02,\n",
       "         2.1583e-01, -3.5234e-01, -1.2197e-01,  3.0294e-01,  7.3882e-01,\n",
       "         1.3297e-01, -3.4018e-01, -8.1062e-02, -2.5973e-01,  1.7507e-01,\n",
       "         2.3974e-02,  4.1746e-01, -6.2069e+00,  1.1619e-01, -5.7523e-01,\n",
       "        -2.6148e-01, -3.7661e-01, -2.8330e-01,  1.3297e-01,  3.6957e-01,\n",
       "         7.4662e-02, -3.4777e-01,  1.5740e-01,  3.4407e-01, -5.0272e-01,\n",
       "         1.7432e-01,  7.9398e-01,  7.3562e-01], device='cuda:0')"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_embeddings(bert_doc, watch1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Dp3LpDtwPIph"
   },
   "source": [
    "## Model III - Preprocessing Embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xafyHRinPIpi"
   },
   "source": [
    "### Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81,
     "referenced_widgets": [
      "51abfd29aa04403e9a2b69eb42609b72",
      "b32b1e7e3d1249cf8ec406631d9ae1a3",
      "aa74920bb1ce4bdabd386ebe58f53abb",
      "1bc31bdb12804834aa1e94ee7fa1b3ab",
      "4ec8c36c8d734fac9b41dfb2d123f5bd",
      "7a0c27699fcb4761a0f78253eb6af0ca",
      "0db028ff95e84d33acdb533760ad41da",
      "846546ae6e8649d5a93bb4012b58856d",
      "b080efa6d7a34395bc08542164b2b6d4",
      "b7d267442de04961a6f52c38d00c2edc",
      "ebacdfc661154ae386dd3ccaeb23f772",
      "9e9badec71f74fdf997043d9a628c597",
      "3c42b1fb09cb44a2aef8cdd4c7961808",
      "c2c41e586b5d47aa93ba8e4dc8675a32",
      "7034dcd30e414f14a4c06c00cc2ade13",
      "2ef29cf73ea14794b8aaf9e3ccc78940",
      "aa43590e7815414fbaea8accf7ae9297",
      "90045b8b64324f729cb4e0873108546b",
      "7e9edc138e6840a4be81c100016ebc6f",
      "16514425790a4386ba23ea1ff1cdda3b",
      "6048025355d34b3ba0bcc2df379dfa05",
      "6a77df41417c4a84adaad20c94b23416"
     ]
    },
    "id": "62H2IcGYPIpi",
    "outputId": "8e1f95d4-464d-4349-b7e1-6e90c2db0e2e"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "51abfd29aa04403e9a2b69eb42609b72",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/3081 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9e9badec71f74fdf997043d9a628c597",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/771 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_dataset_doc = train_dataset.map(lambda row: {'embeddings': get_embeddings(bert_doc, row['sentence'])})\n",
    "test_dataset_doc = test_dataset.map(lambda row: {'embeddings': get_embeddings(bert_doc, row['sentence'])})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "id": "q5rURvw6PIpi"
   },
   "outputs": [],
   "source": [
    "train_dataset_doc.set_format(type='torch', columns=['embeddings', 'labels'])\n",
    "test_dataset_doc.set_format(type='torch', columns=['embeddings', 'labels'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "t9Ov7ucKPIpi",
    "outputId": "18cb714f-5d5c-4273-823b-80113bf887d8"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.2885,  0.3996, -0.3314,  ...,  0.2001,  0.3419,  0.4971],\n",
       "        [ 0.0611, -0.1864, -0.1933,  ...,  0.1184,  0.3338,  0.1873],\n",
       "        [ 0.3170,  0.2567, -0.3027,  ..., -0.0045,  0.7189,  0.4067],\n",
       "        ...,\n",
       "        [ 0.0430,  0.6476, -0.4530,  ...,  0.5743,  0.7295,  0.9185],\n",
       "        [ 0.3391,  0.2675, -0.1105,  ...,  0.0334,  0.3464,  0.4049],\n",
       "        [-0.0085,  0.2371,  0.1148,  ..., -0.2324, -0.0010,  0.2413]])"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_dataset_doc['embeddings']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {
    "id": "OfMQkFCHPIpi"
   },
   "outputs": [],
   "source": [
    "train_dataset_doc = TensorDataset(train_dataset_doc['embeddings'].float(),\n",
    "                                  train_dataset_doc['labels'].view(-1, 1).float())\n",
    "generator = torch.Generator()\n",
    "train_loader = DataLoader(train_dataset_doc, batch_size=32, shuffle=True, generator=generator)\n",
    "\n",
    "test_dataset_doc = TensorDataset(test_dataset_doc['embeddings'].float(),\n",
    "                                 test_dataset_doc['labels'].view(-1, 1).float())\n",
    "test_loader = DataLoader(test_dataset_doc, batch_size=32, shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RDOr-sgLPIpi"
   },
   "source": [
    "### Model Configuration & Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "id": "NEjwk7i8PIpj"
   },
   "outputs": [],
   "source": [
    "torch.manual_seed(41)\n",
    "model = nn.Sequential(\n",
    "    # Classifier\n",
    "    nn.Linear(bert_doc.embedding_length, 3),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(3, 1)\n",
    ")\n",
    "loss_fn = nn.BCEWithLogitsLoss()\n",
    "optimizer = optim.Adam(model.parameters(), lr=1e-3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "id": "93pg9nrOPIpj"
   },
   "outputs": [],
   "source": [
    "sbs_doc_emb = StepByStep(model, loss_fn, optimizer)\n",
    "sbs_doc_emb.set_loaders(train_loader, test_loader)\n",
    "sbs_doc_emb.train(20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 395
    },
    "id": "5jSTsMaSPIpj",
    "outputId": "db5370dc-f3a4-4226-cfbb-295be62459e3"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9EAAAF6CAYAAAAedNt3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAChWklEQVR4nOzddZhUZePG8e+Z2oalu1NKWrpDYumwMF4EwUIxeFVU/OErtigqgoWECAZKSUj3EgJKiUgsnQtbsztxfn+MLK67LFvALNyf69oL9pwzz3lmH2aZe54yoqOjTURERERERETkiizXuwIiIiIiIiIiuYVCtIiIiIiIiEgGKUSLiIiIiIiIZJBCtIiIiIiIiEgGKUSLiIiIiIiIZJBCtIiIiIiIiEgGKUSLiIiIiIiIZJBCtIiIiIiIiEgGKUSLiIiIiIiIZJBCtIiIiIiIiEgGKUSLiIiIiIiIZJBCtPgNp9PJX3/9hdPpvN5Vkb+pTfyL2sP/qE38j9rE/6hN/Ivaw/+oTXIfhWjxKx6P53pXQf5FbeJf1B7+R23if9Qm/kdt4l/UHv5HbZK7KESLiIiIiIiIZJBCtIiIiIiIiEgGKUSLiIiIiIiIZJBCtIiIiIiIiEgGKUSLiIiIiIiIZJBCtIiIiIiIiEgG2a53BURERERE5BKv10tcXNxV2TfY6/XicDg4f/48MTExOV6+ZJ7a5OoKDAwkJCQEiyXn+o8VokVERERE/ITX6+XMmTOEhoZSsGBBDMPI8fKTkpJwOBw5Giok69QmV49pmjidTs6cOUOBAgVy7OerVhK/YpowdWoQ27bpn6aIiIjcfOLi4ggNDSUoKCjHA7TIzcYwDIKCgggNDSUuLi7HylVSEb8RHW3w3HPlefrpPDz4YDA5+O9cREREJFdwOp0EBgZe72qI3FACAwNzdHqEQrT4hfXrrbRtW4AlS/IDsHevlf/+N+g610pERETk2lMPtEjOyunXlEK0XHcrVljp3DmEI0esKY5PmeLghx/s16lWIiIiIiIiqSlEy3XXpImHOnU8aZ574okgDh7Up7EiIiIiIuIfFKLlurPb4bPPEggN9aY6d+GCwaBBwbjd16FiIiIiInLDCw8Pp0uXLtkqY9WqVYSHhzNmzJgcqpX4M4Vo8Qvlynl5442098WLjLTx+usB17hGIiIiInKthIeHZ+pLrqxmzZoUKVLkelfjhqR9osVv9O7tZP58F/PmFUx17p13AmjZ0k3z5mkP+xYRERGR3GvEiBGpjo0fP54LFy6keS4nRUZGEhSUvQVt69WrR2RkJAUKFMihWok/U4gWv/LMM4fYuTOc/ftT/tM0TYPBg4NZvTqWAgXM61Q7EREREbkannvuuVTHvv76ay5cuJDmuZxUuXLlbJcRHBycI+VI7qDh3OJXQkK8fPLJeez21EH52DELjz4ahKkMLSIiInJTOnjwIOHh4QwdOpQ9e/Zw9913U65cOcLDwzl48CAAc+bMYeDAgdSpU4dixYpRunRpOnXqxE8//ZRmmWnNiR46dCjh4eEcOHCATz75hAYNGlC4cGFq1KjB66+/jtebci2fy82JrlmzJjVr1iQ2NpYRI0ZQtWpVChcuTJMmTS5bn4MHD/LAAw9QtmxZSpQoQefOnVmzZg1jxowhPDycVatWZfXHd1lxcXG89tprNGjQgCJFilC2bFn69evH+vXrU13rdDoZN24cTZs2pXTp0hQvXpyaNWty//3389tvvyVf5/V6mTx5Mm3atKFs2bIULVqUatWq0b9//6vyHK4l9USL37n1VjcvveTkxRdTD6v5+Wc7n3/u4MEHk65DzURERETEH+zfv5/27dtTrVo17rrrLs6ePYvD4QDg//7v/7Db7TRq1IiiRYty+vRpfv75Z+677z7eeOMNHnrooQzf56WXXmLNmjV07NiRNm3aMG/ePF5//XVcLhcvvvhihspwu9306tWL6OhoIiIiSEhI4IcffuD+++/n+++/p1WrVsnXHj16lI4dO3L8+HHatWtHrVq12Lt3Lz179qRFixaZ+hlllNPppFu3bmzevJlbb72VoUOHcvLkSWbNmsWSJUv4/PPP6dGjR/L1Q4cOZdasWVSvXp277rqLgIAAjhw5wqpVq/j111+pWbMmAK+88grvv/8+5cqVo2/fvoSGhnL06FHWr1/PihUraN68+VV5PteCQrT4pUceSWLZMhtLl6beJ/qFFwJp3NhN9eqpV/MWERERuZG1bx+SrcebJpimF8OwYOTwLqKLF8flbIHpWL9+Pc8++yzPP/98qnPffvstZcuWTXEsNjaWDh068L///Y8BAwYQHBycofts27aNNWvWULRoUQCeffZZ6taty8SJExkxYkRycE/PsWPHqFOnDnPnzk2+vm/fvnTv3p2PPvooRYgeNWoUx48f58UXX+Spp55KPj5lyhQee+yxDNU5s95//302b95Mv379mDBhAsbf/zAeeugh2rdvz7Bhw2jbti1hYWGcP3+eH3/8kdq1a7NkyRKsVmtyOR6Ph5iYSwsFT548mWLFirFmzZpUP+9z585dledyrShEi1+yWGD8+ASaNbNy6lTKWQeJiQYDBwazdGksGfz9JyIiInJD2LhRb98BihQpwtNPP53muX8HaIDQ0FDuuusuRo4cyZYtW2jWrFmG7vPMM88kB2iAAgUK0LlzZ6ZPn87evXupXr16hsp57bXXUgTuli1bUqpUKbZs2ZJ8LDExkZ9++olChQrx6KOPpnj8PffcwwcffMDevXszdL/MmD59Ona7nZdffjk5QAPceuut3HnnnXz11VfMmzePO+64A8MwME2TwMBALJaU79GtVmuqldPtdnuKoH1Rvnz5cvx5XEuaEy1+q0gRk/HjE9I8t3u3lRdeCLzGNRIRERERf1CjRo3L9gKfOnWK559/noYNG1KsWLHkbbFGjhwJwPHjxzN8n9q1a6c6VqJECQDOnz+foTLy5s2bZrAvUaJEijL+/PNPEhMTqVOnDgEBKbd3NQyDhg0bZrjeGXXhwgUOHDhA+fLlk5/XP10ccn1xrnOePHno0KED69evp0WLFrzzzjts2LABl8uV6rG9e/fm0KFDNG7cmFdffZUVK1aQkJD2e/vcRiFa/Fq7dm4efTQxzXNffhnA7Nn6NFZERETkZlOoUKE0j587d47WrVvz8ccfkz9/fu655x6efvppRowYQefOnQFfj29GhYWFpTp2sWfV48nY1qt58uRJ87jVak2xQNnFodAFC6be7hWgcOHCGbpfZly85+V+nhf3mf7nMO1Jkybx9NNPc+HCBUaPHk3Hjh2pUKEC//3vf4mPj0++7vXXX2f06NE4HA7efvttunfvTrly5RgyZAhnzpzJ8edyLSlEi9976SUntWu70zz32GPBREXl8IQeEREREfFrxmUmdE+ZMoXDhw/zwgsvsGDBAt566y1GjhzJc889R4MGDa5xLTPnYmA/ffp0mudPnjx51e556tSpdO/5zw8TgoODGTlyJNu2bWPr1q2MGzeOihUr8sknn6SYo26z2XjsscdYv349u3bt4rPPPqNx48Z88803DBo0KMefy7Wkbjzxew4HfP55Ai1ahBIXl/IX5vnzvv2j58yJw6Z/zSIiInKDa9Ag7Y6FjLqaC4v5g/379wMk9zr/07p16651dTKlYsWKBAQEsHXrVhITE1MM6TZNk40bN+b4PfPkyUPZsmX566+/OHr0KMWLF09xfvXq1QDJK27/W9myZSlbtix9+vShUqVK/Pzzz4wdOzbVdcWKFaNPnz706tWL+vXrs3z5chISEggKSr0bT26g2CG5QoUKXt56K4GHH069kti6dTbefjuA//4340NzRERERHKj7K6A7fV6SUpKwuFwpFoY6kZQqlQpwLd69z8X/fr2229ZtGjR9apWhgQEBNC9e3dmzpzJ+PHjeeKJJ5LPTZ8+nT/++OOq3PfOO+9kzJgxvPLKK3zyySfJvfy///47X3/9NXny5EneR/v06dOcPHmSatWqpSgjOjqaxMRE8ufPD/iGzG/dupXbbrstxXVxcXHExcVht9tz9b8/hWjJNe6808WyZUl8+23qRSTefDOAFi3cNGmSsbkpIiIiInLj6d+/P2PHjuXZZ59l1apVlCpVit9//50VK1YQERHBnDlzrncV0/XSSy+xfPlyRo0axZo1a5L3iV64cCHt2rXjl19+yVT4dLlcDB069LLnx48fz7Bhw1i0aBEzZszgjz/+oGXLlpw6dYpZs2bhdruZMGFC8nDuo0eP0qJFC2rUqEH16tUpXrw4Z8+eZf78+bhcruRtuBISEujYsSMVK1akdu3alCxZkri4OBYsWMCJEyd47LHHUi2elpsoREuuYRjwzjsJREbaOHgw5S8Pr9c3rHv16hj+tbK+iIiIiNwkSpQowbx583j55ZdZvnw5Ho+HWrVqMWvWLA4fPuz3IbpkyZIsWrSIUaNGsXTpUtasWcOtt97KDz/8wI8//gikvdjZ5Xi9XqZPn37Z8+PHjycwMJDZs2czduxYZs2axccff0xQUBBNmzZl+PDhNG7cOPn60qVL89///peVK1eyYsUKzp49S4ECBbj11lsZMmQI7dq1AyAkJIRXXnmFFStWsG7dOk6dOkV4eDgVK1bk5Zdfpnfv3ln7AfkJIzo62rzelRABcDqdREVFUapUKQIDL7991aZNVm6/PQS3O/VEnm7dXHz1VfwNOcfneshom8i1ofbwP2oT/6M28T9qk8w5derUZVdKzgk3+nDu3CijbXL77bcTGRnJoUOHCA0NvYY1vDHk5GtLrxzJderX9/DCC2nPf549285XX9mvcY1ERERERHJGWvtYz5gxg/Xr19OqVSsFaD+g4dySKw0blsjy5TZWrEj9T/i//w3itts83HKLN41HioiIiIj4r8aNG1OrVi2qVKmC1Wrlt99+Y/Xq1YSFhTF69OjrXT1BPdGSS1ks8Mkn8RQokDooO50GAwcG43Reh4qJiIiIiGTDf/7zH06dOsU333zDp59+yp9//knfvn1ZsmRJihXH5fpRT7TkWsWKmXz8cQL9+4ekOrdzp5UXXwzkrbeUpEVEREQk93jxxRd58cUXr3c1JB3qiZZcrWNHN0OGpD0/+tNPA5g/X58TiYiIiIhIzlGIllzvlVec1KiR9v7QjzwSxNGjWqpbRERERERyhkK05HoBAfDFF/EEB6fere3cOQuDBwfjSTtji4iIiIiIZIpCtPifpKRMP6RyZS+vv56Q5rnVq22MHRuQ3VqJiIiIiIgoRIt/sZ88Sf5OnbBPnpzpxw4Y4KJnz7QD+GuvBRAZac1u9URERERE5CanEC1+w7prF1X/8x/sO3YQ9OST2BYvztTjDQPeey+BUqVSb3vl8fi2vYqOzqHKioiIiIjITUkhWvyCdcUK8nfvTsCJEwAYHg/B99+PZevWTJUTHg6ffRaP1Zp6fnRUlIXhw4MwU58SERERERHJEIVoue6MAwcI6dMHy4ULKY/HxRHSvz/GoUOZKu+22zz8979pb3v1ww8Opk61Z7muIiIiIiJyc1OIluvOLFuWxKeeSvOc5cQJQvr2JbPjsIcPT6RpU3ea50aMCGLvXv3TFxERERGRzFOSEL+QOGIECXfckeY56549hNx9NySm3buc5mOsMHFiPPnypZ4fHR9v8J//BGemOBERERHJ5aZNm0Z4eDjTpk1LcbxmzZrUrFkz2+XkpDFjxhAeHs6qVauu2j0k6xSixT8YBhfeeovzjRqledq2Zg1BjzwC3tSh+HJKlDD58MO0t7367Tcro0YFZqmqIiIiIpKzHnzwQcLDw/nuu+/Sve7ChQsUK1aM0qVLk5CQ9vu83GDVqlWEh4czZsyY612VDLkY6r///vvrXRW/oBAt/sNuZ9+YMbiqV0/ztOO77wh49dVMFdmli5sHH0y7y3n8+AAWLbJlupoiIiIikrMGDBgAwNSpU9O97vvvvychIYE+ffoQFBSUI/eePXs2s2fPzpGycsrgwYOJjIykXr1617sqkgaFaPEr3tBQoqdOxVuiRJrnA999F8eXX2aqzNGjnVSr5knz3MMPB3H8uJHpeoqIiIhIzmnRogVlypRh5cqVREVFXfa6iyH7YujOCeXKlaNcuXI5Vl5OKFCgAJUrVyY4OPh6V0XSoBAtfsdbrBhxM2di5smT5vnAp57CtmBBhssLCoLPP48nMDD13lanT1sYMiQoM6PERURERCSHGYbB3Xffjdfrvexc4127drF582aqV69OnTp1OH/+PGPHjqVz585UrVqVQoUKUbVqVR566CH279+f4Xtfbk70uXPnePLJJ6lUqRLFihWjdevWzJkz57LlTJkyhTvvvJOaNWtSpEgRypYtS69evVi5cmWK68aMGUNERAQAb7zxBvnz56do0aLkz5+fgwcPJl9zuTnRP//8M127dqV06dIULVqUpk2b8uGHH+J2p1xU9+DBg4SHhzN06FD++usv7r77bsqUKUPx4sXp3r07v/32W4Z/RpmV0ToCrFy5kj59+lC1alUKFy5MpUqV6NSpE5MmTUpx3datW7n33nupUaMGhQsXpkKFCrRu3Zq33377qj2Py9FY1hvA7Nmz+eKLL9i6dSvR0dFs27aNMmXKXO9qZYu3enXipkwhpE8fDJcrxTnD6yX4P/8hbt48PHXqZKi8W27xMmaMkyefTD3sZ/lyO+PGORg2LClH6i4iIiJytYS0b5+9AkwTr2liMQwwcnY0Xtzixdl6/F133cXrr7/O119/zYgRIzD+Vb+L4fpiL/Qff/zBa6+9RvPmzenatSvBwcH88ccffPfddyxatIgVK1ZQunTpLNUlPj6eLl26sHPnTho2bEjTpk05cuQI//nPf2jTpk2aj3nmmWeoUaMGrVq1omDBghw9epT58+fTo0cPpkyZQpcuXQBo1qwZhw4dYvr06TRt2pSmTZvi8XiwWq3kzZs33Xp9+OGHjBw5knz58tGnTx+Cg4P5+eefGTlyJOvWrWPq1Kmpfm6HDh2iXbt2VK1alXvuuYf9+/czf/58IiIiiIyMpHDhwln6GeVEHRcuXMgdd9xB3rx56dy5M0WLFuX06dP8/vvvzJgxg/vvvx+A7du307FjR6xWK507d6ZUqVKcP3+e3bt3M2nSJJ5++ukcfQ5XohB9A4iPj6dJkyZ07tyZZ5999npXJ8d4WrYkYdw4gocMSXXOiI8nuF8/YhcvxixbNkPl3X9/EkuX2pgzJ/U+0aNHB9KsmYd69dIe9i0iIiLiD2wbN17vKlw1JUuWpE2bNvzyyy+sXLmSli1bJp9zu93MnDmTgIAA+vfvD0DlypXZs2cP+fLlS1HOypUr6dGjB2+//TYffPBBlury/vvvs3PnTu677z7ef//95OP9+/end+/eaT5m/fr1lP3X+9Ljx4/TunVrXnrppeQQ3bx5cwCmT59Os2bNGDFiBElJSTgcDiyWyw8U3r9/P6NGjaJQoUIsW7aMkiVLAvDiiy/So0cP5s2bx4wZM7jjXzverFmzhlGjRvHEE08kH3v11Vd5++23mTZtGk8++WSGfy5Xktk6Tp06FdM0mTNnTqrRAGfPnk3++4wZM0hMTGTatGnJP8e0rrtWNJz7BnDHHXfw7LPP0qxZs+tdlRznuuMOnCNHpnnOcuoUIX37Ypw7l6GyDAM++CCBEiVSj912uw0GDgziwoVsVVdEREREsuFyC4wtWLCAkydP0rlz5+TQnDdv3lQBGnzzq6tWrcry5cuzXI9vvvkGh8PB888/n+J427ZtU4T7f/p3gAYoWrQoERER7Nu3j0OHDmW5PgDffvstbrebRx99NDmcAgQEBDBq1CgAvv7661SPK1OmDI8//niKYxd/zlu2bMlWnXKqjmktEpc/f/4sX3e15doQPXbsWMLDwwkPD2fjdfhEbsaMGTzxxBO0atWKwoULZ2ivuC1bttC3b19Kly5N8eLFadeuHbNmzbpGNc69Ep96iqR7703znHXvXoLvuguczgyVlS+fycSJ8VgsqedHHzhg5emnc2aVRxERERHJvM6dO1OwYEHmzp3L+fPnk49fbkGxVatWcdddd1GlShUKFiyYnA927tzJ8ePHs1SHCxcucPDgQcqXL0+RIkVSnW/cuHGajztw4ACPP/44tWvXpkiRIsl1mThxIkCW63PR9u3bAdLsOGvYsCGBgYFpznOuWbNmqh7uEn8v4vvPn3FOyGwdL/bqt2vXjmeeeYY5c+Zw5syZVI/t2bMnFouFe+65h0ceeYTvvvuOo0eP5mjdMyNXhuidO3cyZswYQkJCrlsdXn31VSZNmkRUVFSaL65/W7lyJR07dmT9+vX07NmTBx54gBMnTvDAAw8wbty4a1DjXMwwSHj3XVzt2qV52rZuHUFDh2Z4D+mmTT0880za217NnOngm29SD/cWERERkavPbrfTv39/EhISkveMPnHiBL/88gslS5akVatWydf++OOPdOvWjVWrVtGoUSOGDh3Ks88+y4gRIyhVqhRJSVlb7yYmJgaAggULpnk+rTnEf/31F61atWLatGmULVuWBx54gGeeeYYRI0bQtGlTABIT037/mdl6FSpUKNU5wzAoVKhQ8jX/FBYWluqYzeab1evx5OxUxszWsUePHkybNo1q1arxxRdfMGDAACpWrEi3bt2SAzlA/fr1mTt3Lk2aNOG7777jwQcfpFq1arRp0ybVwm3XQq6bE+1yuRg6dCg1a9akfPnyzJw5M8OP/eabb2jSpMllFxjweDyMHz+ewYMH43A40i1r3LhxlC9fntKlS/Pee+/xyiuvXPZat9vNsGHDsFgszJs3j1q1agHw7LPP0rZtW0aPHk337t1T1GvUqFGMHTs23TpER0ene/6GYrMR/+WXhHbpgvUfL6iLHLNmYZYqhfP//i9DxT3zTCIrV9pYty71S+Dpp4No0MBDhQpasltERET8i7tBg+wVcBUXFsspAwYM4KOPPmLKlCkMHDiQGTNm4Ha7ufvuu1P0qL7++usEBgayfPlyKlSokKKMH374Icv3vxg6T58+neb5kydPpjr28ccfEx0dzYQJE5LnbF/05JNPsmbNmizX59/1OnXqVKo8Y5omp06dSjMwX0tZqWOXLl3o0qULMTExbNiwgTlz5jBlyhT69OlDZGQk4eHhADRp0oQmTZqQkJDApk2bWLBgAZ9//jn9+/dn3bp1aQ6nv1pyXYh+++232b17NytWrEgxyf9Kjhw5wrBhwyhSpAhz585N1aher5ehQ4cyc+ZMHA4HgwcPTre8f34KdiUrV65k//793H333ckBGnzzOIYPH87DDz/M9OnTGTFiRPK5Rx99lLvuuivD97gphIURN3Mmoe3aYTl8ONXpgA8+wFuqFEmDBl2xKJsNJk6Mp1mzMM6fT/kfSGysb370okVxXOGzFBEREZFrKrsrYHu93gwtYnU9Va1alQYNGrBx40Z+//13pk2blrwF1j/t37+fqlWrpgrQx48f58CBA1m+f548eShTpgx//fUXJ06cSDXqdN26dakec3FLrc6dO6c4bpomGzZsSHW91WoFMtcTXKtWLebOncvq1aupV69einObNm3C6XTSsGHDDJd3NWSnjmFhYbRr14527drh8XiYOnUqmzdvpm3btimuCwoKonnz5jRv3py8efPy2muvsWzZMh544IGr9rz+zT9fOZexdetW3nnnHUaMGEHVqlUz9dgSJUrw2WefcfToUSIiIlJs4u71ehkyZAgzZ87kjjvu4MEHH8zReq9evRogzeXwL/6j+PenUwULFqRy5crpft2MzKJFifvuu8vvIT1iBLb58zNUVqlSJh98EJ/mua1bbbz6amCW6ykiIiIiWXdx7vPTTz/Nnj17aNWqVapOsFKlSrF///4UPcNOp5Phw4fj+tcWqZnVv39/kpKSeO2111IcX7p0KStWrEh1falSpQDfCt3/9N5777Fz585U119cEO3IkSMZrlPfvn2x2Wx89NFHHDt2LPl4UlISL7/8MsB174TLbB3XrFmT5gcJp06dAnwLkgFERkbiTGMNpH9fd63kmp7oxMTE5GHcw4YNy1IZERERfP755wwcOJCuXbsyd+5cSpQokdwD3a9fPz7++OMc/1Ru3759AKk+JQMoUqQIoaGh/PXXX1ku/9y5c0RFRSV/ArZnzx7Onz9PqVKl0lyxMLfzVq1K3LRphPTujfGvuS6G10vwwIHEzZ2L51+ffqWle3c399+fyKRJqV94H3wQQKtWbtq0Sb0pvIiIiIhcPT179uS5555LDqX/XlAMYPDgwTz77LO0aNGCbt264fF4WLZsGaZpUqNGDX7//fcs33/YsGHMnTuXr776it27d9OkSROOHDnCrFmz6NixIwsXLkxx/QMPPMC0adO499576dGjB/nz52fTpk1s27YtzesrV65MsWLF+OGHH3A4HBQpUgSbzcZDDz102b2iy5Urx6hRoxg5ciRNmzalZ8+eBAcHs2DBAvbu3Uvnzp1TDSXPaZ9//jm//PJLmufuvfdeGjdunKk6jhgxguPHj9OoUSNKly6NYRisX7+ezZs306BBg+RF3MaOHcvq1atp3LgxZcqUITAwkG3btrFixQrKli1L165dr+rz/rdcE6Jfe+019u3bx/Lly5OHP2RF9+7d8Xg8DBo0iIiICOrUqcOsWbPo3bs348ePvyrDWi78vW9Snsv0noaFhSVfkxXz58/nkUceSf6+X79+AHz00Uephr3cKDzNm5Pw0UcEpzF020hIILh/f+IWL8ZbrtwVy3rtNSfr1tnYsyf1v6shQ4JYvTqWwoVTr+YtIiIiIldHWFhY8qJT+fLlS7U3MMCgQYOw2+1MnDiRyZMnkzdvXjp06MDLL7/Mfffdl637h4SEMG/ePF555RXmzp3Ltm3bqFq1Kl988QUXLlxIFYpvvfVWfvjhB/73v/8xd+5cLBYLt912GwsWLODnn39Odb3VamXKlCm8/PLLfP/998TGxgK+HvDLhWjwTfksX748H330ETNnziQpKYkKFSrw6quvMmTIEIyrPM997dq1rF27Ns1zzZo1o3Hjxpmq4/Dhw5kzZw5bt25l6dKl2Gw2SpcuzSuvvMLAgQOTc9/AgQPJkycPmzdvZu3atZimScmSJXnqqad4+OGHL5uzrhYjOjra79NBZGQkt99+O//973959tlnk48PHTqU6dOns3jxYhpkcpGFmTNnJs977tKlC5MnT85yOL+4sNjlQmvPnj1ZtmwZW7ZsoXz58qnO33LLLcTFxWV777jczul0EhUVRalSpQgMzNhQ6oD33iPwMou6eSpUIG7RIswCBa5Yzu+/W2jbNpTExNS/eNq1czFzZjx+Om3oqspKm8jVo/bwP2oT/6M28T9qk8w5depUmisb55TcMCf6ZqM2uTZy8rXl963kdrsZOnQo1atX58knn8yRMk3TTLEU+u7du9NcZS+nXPxk5HK9zTExMdf805MbReITT5D4n/+kec66b59vD+mEhCuWU6OGl9Gj095r+pdf7Hz8sVYYExERERGRXBCiY2Nj2bdvH7/99huFChVK3rQ8PDyc6dOnA9C+fXvCw8OZO3fuFcszTZNhw4YxdepUevXqxcSJEzlw4AAREREpJr/npItzoS/Ojf6nEydOEBsbm2YPtWSAYeB8801cHTumedq2YQPBDz2UoT2kBw1K4vbb016E4pVXAtm61e9fLiIiIiIicpX5/ZzogICANBcSAN+Y/H379tGpUycKFix42f2fLzJNkyeeeILJkyfTs2dPPv30U6xWKxaLhYceeoiIiAjmzp1L0aJFc/Q5NG3alHfffZelS5fSu3fvFOeWLFmSfI1kkc1G/OefE9K1K7atW1Odts+eTeCLL+L83//SLcYw4KOPEmjWzMqxYykDs8tlMHBgMCtWxBIampOVFxERERGR3MTvQ3RQUBDjxo1L89zQoUPZt28fw4cPv+KcaNM0GT58OF999VWKAA3Qp08fgBRB+t/7wWVHy5YtKVu2LN999x0PPfRQ8l7R58+f591338XhcHDHHXfk2P1uSqGhxM+YQWj79ljSmFse8NFHvj2khwxJt5gCBUwmTIine/cQTDPl/Oh9+6w8+2wQH3985eHhIiIiIiJyY/L7EJ1Tjh07xty5c+nRoweffvopNlvKp96nTx9M02TIkCEsX778isvDT548OXmj9Yt7v02ZMiV5T+jGjRtz7733AmCz2fjggw/o3bs3Xbp0oVevXoSGhjJ79myioqIYPXo0ZcqUyemnfNMxixQh7rvvCOnQAUt0dKrzgc89h7dECdwREemW06KFh6eeSuTtt1MvfvL11w7atHHTp0/29h4UEREREZHc6aYJ0cWLF2fx4sWULFkyVYC+qG/fvtSpU4eKFStesbx169Ylz8m+aP369Sk2WL8YogFatGjBggULGDNmDLNmzcLlclGtWjVeeeUVevXqlcVnJf/mrVyZ+K+/JqRHj9R7SJsmwYMGETd7Np6GDdMtZ8SIRFassLFxY+p/K08+GYTdbtKtm5urvIuAiIiIiIj4mVyxxZXcHHJyCwz7Dz8QfJlVu7358/v2kP57wbfLOXDAoEWLMC5cSDspt23r4s03nVSocOVFy3IrbUviX9Qe/kdt4n/UJv5HbZI52uLq5qM2uTZuqi2uRLLC1asXCaNHp3nOcvYswX36YJw+nW4ZZcuajB17+fnPS5bYadw4lFdfDSA+PlvVFRERERGRXEIhWm5YSY8+SuKgQWmes+7fT/Add3Cl9Nurl4t77km67PmkJIO33w6kUaMw5s+/aWZHiIiIyFVkmhooKpKTcvo1pRAtNy7DwPn667g6dUrztG3TJoIHDwaPJ91i3norgc6d019I7NAhC3fdFUL//sEcOKCJ0iIiIpI1gYGBOJ3O610NkRuK0+nM0ekkCtFyY7Naif/8c9z16qV52j53LoHPPw/pfDoVFATTpsXz6afxFCmS/vznhQvtNGoUxhtvBKD//0RERCSzQkJCiI2NJSEhQT3SItlkmiYJCQnExsYSEhKSY+Vq/Knc+IKDif/mG0Lat8d64ECq0wETJuAtXZqkRx65bBGGAX37uujQwcWYMYFMnOjA6027x9npNBgzJpAZM+y8+aaTdu3cOfVMRERE5AZnsVgoUKAAcXFxnL7C+i1Z4fV6k3vltIiVf1CbXF2BgYEUKFAgR3+2CtFyUzALFSL+u+8Iad8ey7lzqc4HjhyJt2RJ3N27p1tO3rzw+utO7r47iaefDmLDhsu/hP76y0qfPiFERLh47bUESpXSp8kiIiJyZRaLhbCwMMLCwnK8bKfTyYULFyhSpIhWS/cTapPcRx91yE3DW7Ei8dOnYwYEpDpnmCbBgwdj/cc+3+mpWdPLzz/H8dFH8RQsmP4Q7zlz7Nx2WxjvvRdA0uXXKBMRERERkVxAIVpuKp5GjYifOBHTSD0U20hMJPjOO7Hs3ZuhsiwWuPtuF5s2xTBwYCKGcfme5vh4g1deCaRZs1BWrLBmuf4iIiIiInJ9KUTLTcfdvTvOV19N85zl3DlC+vTBOHkyw+WFh8M77zhZtiyWevXSn//8xx9WuncPZeDAII4e1SreIiIiIiK5jUK03JSSHn6YxIceSvOc5eBB3x7ScXGZKrN2bS+LF8cxdmw8+fKlP8T7++8dNGwYxocfOnClv3uWiIiIiIj4EYVouTkZBs7XXsPVtWuap21bthA8cOAV95D+N4sF7r/fxaZNsQwYkP4E6NhYg5Ejg2jZMpQ1azTEW0REREQkN1CIlpuX1Ur8p5/ibtAgzdP2BQsIHDEi3T2kL6dAAZNx4xJYvDiWWrXSD+I7d1rp0iWUhx4K4uRJDfEWEREREfFnCtFycwsKIn76dDzly6d5OuCzz3CMG5fl4hs08LBsWSxvvplAnjzph/EZMxzUrx/GxIkO3NpaWkRERETELylEy03PLFiQ+O++w1ugQJrng156CfsPP2S5fKsVBg9OYtOmGO64I/0h3hcuGDz7bBCtW4eycaOGeIuIiIiI+BuFaBHAW7488d98g3mZDe6DhgzBumZNtu5RuLDJJ58kMH9+LNWqpT/E+7ffrLRvH8pjjwVx5oyGeIuIiIiI+AuFaJG/eRo0IP7TT9PeQzopiZC77sKyZ0+279OkiYcVK2L53/8SCA1Nf4j3lCkO6tUL5csvHXjTX/BbRERERESuAYVokX9wR0TgHDMmzXPG+fO+PaRPnMj2fex2eOSRJDZujKF37/SHeEdHW3jyySDatQvh1181xFtERERE5HpSiBb5l6QhQ0h85JE0z1miogjp0gXHxIkYJ09m+17Fipl8/nkCP/0US+XK6Q/x3rLFRps2ITz1VCDR0dm+tYiIiIiIZIFCtEganKNH4+rePc1z1j//JOjZZwmrWpXgnj2xT5lCdlNty5YeVq+OZdSoBIKDLz/E2zQNPv88gHr1wpg61a4h3iIiIiIi15hCtEhaLBbiJ0zA3ajRZS8xvF7sy5YR/Nhj5KlcmeC77vKt4h0Xl6VbOhzwxBNJbNgQQ7durnSvPXPGwqOPBtOpUwjbt+tlLCIiIiJyrejdt8jlBAYS//XXeCpUuOKlRlIS9vnzCf7Pf8hTuTJBDz6I7eefISn9+c5pKVXKZPLkeL77Lo7y5dMf4r1hg41WrUIZMSKQ8+czfSsREREREckkhWiRdJj58xP3/fd4qlTJ8GOMuDgc331HyJ13kqdSJYIeewzrihXgST8Q/1u7dm7Wro3l+eedBAZefoi312swYUIADRuGMXOmHTP9Bb9FRERERCQbFKJFrsAsW5bYFStIeOst3LfdlqnHGufP45gyhdDu3Qm75RYCn30Wa2QkGU26gYHw7LOJrF8fQ8eO6Q/xPnHCwuDBwXTuHMLixTbNlxYRERERuQoUokUyIjCQpEGDiFu4kAvbt5Pwyit4atbMVBGWkycJmDiR0A4dCKtVi4BRo7D89luGAnXZsiYzZsQzfXocpUunn47XrbPRt28IjRr59peOj89UNUVEREREJB0K0SKZZJYuTdKwYcSuWkVMZCTOESPwVKyYqTIsUVEEjh1LWPPmhDZqRMCbb2LZt++Kj+vUyc369TE8/bQThyP98P3HH1aefDKI6tXDGD06gGPHjEzVUUREREREUlOIFskGb+XKJD73HLEbNxKzfDmJjz+Ot2TJTJVh3bOHwNdeI6xePUJatcIxbhzGkSOXvT44GEaOTGTduljatk1/iDfAuXMW3nknkFq1whg8OIitW/WyFxERERHJKr2bFskJhoG3dm2c//d/xGzfTuzPP5P44IN4CxbMVDG2rVsJevFF8lSvTkinTjg+/xzj9Ok0r61Qwct338Xz1VdxlChx5QnQLpfBzJkOWrUKo1OnEObMsWV2rTMRERERkZueQrRITrNY8DRujPPtt4nZvZu4H34g6a67MPPkyVQxtnXrCHrqKcKqVCG4Tx/sX38NFy6kuMYwoHt3Nxs2xDBqVALFi2dsNbF162wMGBBCvXqhjB/vICYmU1UTEREREblpKUSLXE02G+42bUj4+GMu/PEHcVOnktSzJ2ZQUIaLMDwe7L/8QvDDD5OnUiWCBwzA9uOPkJCQfE1oKDzxRBLbtsXw2Wfx1K3rzlDZBw5Yee65IKpXz8MLLwRy8KDmTYuIiIiIpEchWuRaCQzE3bUrCV9+yYW9e4n/9FNcHTti2mwZLsJITMQ+Zw4h99/v24N68GBsixaByzc32m6HPn1cLFkSx4IFsXTr5sJiufLq3xcuGHz0UQB16oRx333BbNhg1X7TIiIiIiJpUIgWuR5CQ3H17Uv8jBnE7N1L/Pvv427RAtPIeE+wERuLY+ZMQvr1I6xyZQKfeALrqlXg8WAY0KiRh8mT49myJYaHH04kLOzKqdjrNfjpJzsdO4bSrl0IP/4YgNut3mkRERERkYsUokWuMzNfPlz33Ufc7NnE7NpFwpgxuOvXz1QZlnPnCJg0idCICELr1fMN9/67K7lsWZPXXnOyY8cFXnstgTJlMjZvevNmG0OGhNO9e00+/DCY6OhMPjERERERkRuQQrSIHzGLFiVp6FDifvmFC1u34nzpJTzVqmWqDOuBA4Tcfz8hXbpg2b49+XiePPDww0ls2RLD5MlxNG6csXnTJ086ePXVMKpXz8MzzwSyb59+bYiIiIjIzUvvhkX8lFm2LInDhxO7di0x69bhfPppPOXKZfjxtrVrCW3ZkqBhwzBOnUo+brVCt25ufv45jmXLYunbNwmb7cpDvePiDD79NID69UO5445gVq7UvGkRERERufkoRIvkAt5bbiFx5Ehit2whdulSEh95BG/x4ld8nGGaOL76irB69XB8+CEkJaU4X6eOh08/TWDbthiefNJJePiVh3qbpsGCBXa6dQulRYtQvv7aTmJilp+aiIiIiEiuohAtkpsYBp66dXH+73/E/P47sXPnkvjAA3jz50//YRcuEDRyJKFNmvhW8/6XEiVMXn45kR07YnjnnQQqVvRkqDq//Wbl4YeDqVUrjDffDOD0aS1CJiIiIiI3NoVokdzKYsHTrBnO994jZs8e4idMuGLvtPXPPwnp14/gPn2w7NmT6nxICAwcmERkZCwzZsTRvHnGuphPnLDw2muB1KgRxuOPB7Frl361iIiIiMiNSe90RW4Edjuu/v2J2bgR5zPPYAYGpn/5L78Q2rQpgc89R1rLblss0LGjm2+/jebrr3dwxx0JOBxXngDtdBpMnuygceMwevUK5pdfbJo3LSIiIiI3FIVokRtJSAiJL7xATGQkST17pnup4XYTMH48YXXr4vjiC/CkPYS7UqUExo69wO+/xzBihJOCBTO2RdbSpXb69AmhUaNQJk2yk5CQ6WcjIiIiIuJ3FKJFbkBm6dIkfPklsfPm4alZM91rLWfPEjR8OKEtWmBdufKy1xUubPLcc4n8/nsM48bFU61axuZN79lj5YkngqlePYxXXw3g+HHNmxYRERGR3EshWuQG5mnalNjly4l//328BQume611xw5Cu3UjeMAAjAMHLntdYCAMGOBizZpYfvwxlg4dXBmqy9mzFt5+2zdvesCAYBYtsl2u81tERERExG8pRN8AZs+eTY8ePShbtizh4eEcPHjweldJ/InViuu++4jZvJnERx/FtNnSvdw+Zw5ht91GwOjRGHFxl73OMKBVKw8zZ8YTGRnDf/6TSFDQlSdAu90Gc+bY6dcvhJo1fb3TBw6od1pEREREcgeF6BtAfHw8TZo04fnnn7/eVRF/ljcvzldfJXbdOlwdO6Z7qZGYSOA771CgSRMKzJsH3vTnQVeu7OXdd53s2BHDSy85KVYsY/Omjx719U7Xrp2Hbt1C+PZbzZ0WEREREf+mEH0DuOOOO3j22Wdp1qzZ9a6K5ALeSpWInzGDuO++w1O5crrXWk+coNyoUeTv0gXrpk1XLDt/fpPhwxPZti2GiRPjqV3bneF6rVxpY9CgYKpWzcMzzwSybZt+PYmIiIiI/8kV71KdTifPP/88nTp1omrVqhQpUoTKlSvTsWNHpk6disuVsTmZOWnGjBk88cQTtGrVisKFCxMeHs60adPSfcyWLVvo27cvpUuXpnjx4rRr145Zs2ZdoxqLpORu147YNWtIeP11zLx5073W/uuvhLZrR9BDD2EcPXrFsh0O6NfPxbJlccyfH0vXri4MI2N7XZ0/b/DppwG0bBlGixahfPqpI61duERERERErotcEaLj4uL44osvMAyDDh068Mgjj9C1a1eOHj3Ko48+Sv/+/fFeYbhpTnv11VeZNGkSUVFRFClS5IrXr1y5ko4dO7J+/Xp69uzJAw88wIkTJ3jggQcYN27cNaixSBrsdpKGDCFmyxYSBw7EtKT/K8ExYwZhDRoQ8M474HResXjDgCZNPEydGs+vv8bw9NNOihfP+Gt1+3YrzzwTRNWqeRg0KIgVK6xXGlkuIiIiInJV5YoQnS9fPg4dOsT8+fP54IMPeOmll3j33Xf59ddfadasGUuXLmXx4sVXLOebb77h0KFDlz3v8Xj48MMPSUpKumJZ48aNY/v27ezbt4///Oc/6V7rdrsZNmwYFouFefPm8f777/O///2P1atXU7FiRUaPHp2qXqNGjSI8PDzdL5GcYhYogPOdd4hduRJ38+bpXmvExRE4ejRhDRti++knMDPWw1y2rMnIkYn89lsMM2fGERHhwmbL2GOdToNvv3XQvXsodeuG8vbbARw9qsXIREREROTayxUh2mKx4HA4Uh232Wx07doVgL/++ivdMo4cOcKwYcPo2rVrmkHa6/UydOhQRo4cyaRJk65Yp1atWlG6dOkM1X/lypXs37+fPn36UKtWreTjefPmZfjw4SQlJTF9+vQUj3n00UeJjIxM90skp3lr1CBu9mziJk/GW6ZMutdaDh0i5L77COnaFctvv2X4HlYrdOjgZsqUeHbtimH06ASqVMn4XlcHDlh59VXfVln9+gUze7aNDHzuJSIiIiKSI3JFiL4cr9fLkiVLAKhWrVq615YoUYLPPvuMo0ePEhERQVRUVIpyhgwZwsyZM7njjjt48MEHc7Seq1evBqBNmzapzrVt2xaANWvWpDhesGBBKleunO6XyFVhGLi7dSNmwwZinn8eT1BQupfb1qwhtGVLAp98EuP06UzdqlAhk8ceS2L9+lgWLYplwIAkQkIy1jvt9RosWmTn3ntDqFYtjJEjA9mzJ1f/ShMRERGRXCBXveNMSkpizJgxvPbaazzzzDM0atSIxYsXc/fdd9OyZcsrPj4iIoLPP/+cI0eO0LVrV6KiopJ7oGfOnEm/fv34+OOPsVxhXmhm7du3D4AKFSqkOlekSBFCQ0Ov2JOennPnzrF9+3b27t0LwJ49e9i+fTvnzp3LcpkiBAYS//jj/P799yT07ZvupYbXS8CXXxJWty6Ojz4is13DhgENG3oYNy6BPXsuMG5cPLfdlvGVvU+ftvDhhwHcdlsYHTqEMHmyndjYTFVBRERERCRDcl2IfuONN3jzzTf59NNP2bt3L4899hjvv/9+hsvo3r07EydO5PDhw0RERDBw4EBmzJhB7969GT9+fI4HaIALFy4AkCdPnjTPh4WFJV+TFfPnz6dFixbcd999APTr148WLVowf/78LJcpcpGrUCEujBtH7JIluBs0SPda48IFgl54gdCmTbFlYJ2CtISGwoABLhYujGPDhhgeeyyRggUzvppYZKSNxx/3bZX12GNBREZaMzptW0RERETkinJViA4NDSU6OpqzZ8+yY8cO3n77bSZPnkzXrl0zFUJ79erF+PHjOXDgALNmzaJLly5MnDgRq9V6FWt/9dx9991ER0en+rr77ruvd9XkBuKpV4+4hQuJnzgRb7Fi6V5r3buXkL59Ce7XD8vfIySyokoVL6NHO9m5M4bJk+Po0MGFxZKxRBwbazBlioMOHUJp1CiUceMcnDqlxchEREREJHtyVYi+yGKxUKJECQYOHMj777/P+vXreeeddzL8eNM0WblyZfL3u3fv5uTJk1ejqsClHujLBf2YmJjL9lKL+BWLBVe/fsRs3Ijz6acxAwLSvdy+aBGhjRsT+PzzZGezZ4cDunVzM3NmPL//HsPIkU7Kls34YmR79lh58cUgbrkljAEDglm0yIYn4w8XEREREUmWK0P0P7Vu3Rq4tHjXlZimybBhw5g6dSq9evVi4sSJHDhwgIiICI4dO3ZV6nhxLvTFudH/dOLECWJjYylfvvxVubfIVREaSuLIkcRERuLq3j3dSw23m4CPPyasXj3skyaR3fRavLjJ008nsmVLLLNnx9KvXxKBgRnrnXa7DebMsdOvXwg1a4bx6qsBHDig3mkRERERybhcH6KPHz8OgN1uv+K1pmnyxBNPMHnyZHr27Mmnn35Kv379mDBhAvv37yciIiK5vJzUtGlTAJYuXZrq3MXVxS9eI5KbmGXKEP/VV8TOnYunRo10r7WcOUPwE08Q2qwZAaNGYZs7FyMbI0AsFmjRwsPEiQns3n2Bt99O4NZbMx7Qjx618PbbgdSunYeIiBBmzrSTkJDl6oiIiIjITSJXhOjdu3cTHx+f6nh8fDwvvPACAO3bt0+3DNM0GT58OF999VVygL44B7pPnz4pgvSJEydytP4tW7akbNmyfPfdd2zfvj35+Pnz53n33XdxOBzccccdOXpPkWvJ06wZsStWED92LN4CBdK91rprF4FjxxJyzz3kqVyZsFtvJejBB3FMmID111/B5cr0/cPD4cEHk1ixIpaVK2MYNCiRvHkzvprYqlU2Bg/2LUb29NOBbN6sxchEREREJG22612BjJg1axYff/wxjRo1onTp0oSFhXH06FF++eUXzp49S+PGjXn44YfTLePYsWPMnTuXHj168Omnn2KzpXzqffr0wTRNhgwZwvLly+nfv3+65U2ePJl169YBsHPnTgCmTJmSPKy8cePG3HvvvQDYbDY++OADevfuTZcuXejVqxehoaHMnj2bqKgoRo8eTZkyZbL0sxHxG1Yrrvvvx9WjB4Fvvolj4kQM95W3qbIcPIjj4EH47jsAzMBAPHXq4GnQAHf9+ngaNsQsWjTD1ahVy8tbbzkZPdrJ3Ll2pkxxsGJFxn7VnT9v8NlnAXz2WQAVKnjo08dFv34uKlTI+OrgIiIiInJjM6Kjo/2+v+XXX39l0qRJREZGcvToUeLi4siTJw/Vq1end+/e3HPPPalCcVoOHDhAyZIl0732zz//pGLFilcsa+jQoUyfPv2y5++8807Gjx+f4tjmzZsZM2YMkZGRuFwuqlWrxiOPPEKvXr2ueL+bgdPpJCoqilKlShEYGHi9qyNkr00se/cS+MIL2BctynY9vKVK4W7QAE+DBngaNsRTs6ZvtbEMOnDAYNo0B19/7eDIkcwPwKlb103fvi569XJRpMj1+5Wp14j/UZv4H7WJ/1Gb+Be1h/9Rm+Q+uSJEy81Bv0D8T060iW3xYgKffx5rNra6+jczIABP7dq+3uq/w7VZvPgVH+fxwNKlNqZMcfDzzzZcrswtKmaxmLRs6QvUXbu6uNaL6us14n/UJv5HbeJ/1Cb+Re3hf9QmuU+uGM4tIrmXu317Ylu1wv7119hnz8a2cSNGJvZ1T4uRmIhtwwZsGzZwcZMtb4kSl3qrGzTAc+ut8K8tuKxWaN/eTfv2bk6fNvjmGztTpzrYvTtje8R7vQbLltlZtszO8OEmnTq56NvXRbt27sx0jIuIiIhILqYQLSJXn92O6777cN13H3i9WPbuxRoZiW3TJqwbN2LZtQsjmyt5WY4cwXHkCPz4IwCmw4Hn1lvx/D2v2t2gAWaJEmD4ep8LFjR59NEkHnkkiU2brEyZ4mDWLDsxMRnrnXY6DWbNcjBrloN8+bz06OEL1I0aebDkiiUbRURERCQrFKJF5NqyWPBWqYK3ShVcAwb4jp0/j/XXX7FFRmLduNEXrKOjs3UbIykJ28aN2DZuhL/XJ/AWK5ZiCLindm2MwEAaNPDQoEECb7yRwMKFNmbOdLB4ccaHe587Z+HLLwP48ssASpb00qdPEn37uqheXQuSiYiIiNxornqIjo6O5ujRo1SoUIGAfw2tFBEBIG9ePK1a4WnVyve9aWL580+skZFYN23CFhnp6632Zi+UWo4dwzJ7NvbZs323sdvx1KyZvGCZUb8+PbqXpkcPN+fOGfz0ky9Qr12b8V+Vhw9bGDs2kLFjA6lWzUO/fkn06eOiZEktPyEiIiJyI8h2iN62bRtz586lcePGtGnTJvl4QkICjz76KLNmzQIgb968jB07lu7du2f3liJyozMMvJUq4a1UCdfdd/uOxcRg3bIF29891daNG7GcPZu927hc2LZswbZlC0yYAPh6q109e1Lg/vu5//7K3H+/i6gog++/t/Pttw527MjY/GmAnTutjBoVxKhRQTRp4qZfvyS6d3eTL58CtYiIiEhule2Ze1OnTuWdd97B/Nd8xtdee40ffvgB0zQxTZPo6GgGDRqUvKeyiEimhIXhadmSxKefJn7GDGL27SNm82biP/mExIED8dSsiZkDk5Etx44R8PHHhDVsSEiXLti//55ShZ088UQSa9bEsnZtDE8+6aRkycz1iq9da+OJJ4KpXDmMu+4K5scfbSQkZLu6IiIiInKNZfsd59q1awkMDKR169bJx5KSkvjqq6+w2+3MnDmTAwcO8NBDD+Fyufjkk0+ye0sREV9vdYUKuO64A+c77xC7ahUXDh0ids4cnC+9hKtTJ7wFC2brFrY1awgeOJCwatUIfOklLH/9RbVqXl5+OZHt22OYPz+WBx5IJF++jAdql8tg/nw7998fQuXKeXj44SCWL7fi8WSrqiIiIiJyjWQ7RJ88eZJixYph+UcPUGRkJDExMXTq1In27duTN29eXn75ZUJCQlizZk12bykikrbQUDzNm5M4fDjx06cTs3cvMb/+SvzEiSQOGoS7dm1Ma8aHY19kOXOGgA8+IKxuXYJ79MD2009YPC6aNPHw3ntO9uyJ4euv4+jZM4nAwIwP1Y6JMfj6awc9eoRSrVoYzz0XyK+/WsnmQuUiIiIichVle050dHQ0ZcqUSXEsMjISwzBo27Zt8rGgoCDKli3Lvn37sntLEZGMMQy85crhLVcOV79+vmPx8Vh//TV5wTLrxo1YTp7McJH25cuxL1+Ot0gRku65h6R778VRpgydO7vp3NlNTEwCc+fa+fZbO8uX2/B6M7bC94kTFsaPD2D8+AAqVfLQp4+Lfv1clCunFb5FRERE/Em2e6KDgoI4ffp0imPr1q0D4Lbbbktx3OFwpOixFhG55oKD8TRtStKwYcRPm0bMnj1c2LaN+M8+I6l/f8wM7iJgOXGCwHfeIax2bYL79sU2bx643YSFwZ13uvjhh3h27YphzJgE6tZ1Z6qKe/daGTMmkDp1wmjXLoQJExycOpWxMC4iIiIiV1e2E23lypU5dOgQu3btAuDMmTOsWrWKAgUKUKVKlRTXHjt2jILZnKMoIpKjDAOzTBlcffqQMGECMbt3k/Daa3gqV87Yw00T++LFhNx9N2G1ahEwZgzGkSMAFCliMnRoEkuXxrFpUwwjRjgpXz5zk583bbIxYkQQVauGceed4cyfn5/YWAVqERERkesl2yG6R48emKZJ3759eeGFF4iIiCApKYlevXqluC4qKorjx49Tvnz57N5SROSqMfPlI+nhh4ndsIHYuXNJ6tMH027P0GMtR48S+MYbhNWsSfAdd2BbtIiLK4ZVrOjluecS2bw5liVLYhkyJJHChTM+VNvjMVi2LICXXy5P9eqF6N8/mMmT7Zw+rUAtIiIici1lO0QPHjyYJk2acOTIET7++GN27dpFxYoVGTFiRIrrLu4X3bx58+zeUkTk6jMMPM2akfDZZ8Ts2kXC//0fngx+CGh4vdgXLCCkXz/Cbr2VgLfewjh+/GKx1Kvn4fXXnezcGcMPP8Rxxx1JhIZmfDWxxESDhQvtPP64b8usLl1CGD/ewaFDCtQiIiIiV5sRHR2d7XVgvV4vP//8M3v37qVUqVJ06dKFwMDAFNd89NFHHD58mAcffJAKFSpk95ZyA3I6nURFRVGqVKlU/37k+lCb/IvXi3XlSgK+/BLbvHkY7ozPdTatVtydO5P0wAO4W7WCf60PER8PCxbYmTnTzi+/2HC7sxaIb73VQ9euLrp2dVG1qhdDufqq0mvE/6hN/I/axL+oPfyP2iT3yZEQLZIT9AvE/6hNLs84cQLHtGk4Jk3CcuhQph7rKVuWpPvvx3X33ZiFCqU6f/aswY8/+lb4Xrcu65soVKjgoWtXN127uqhXz/Pv3C45QK8R/6M28T9qE/+i9vA/apPcR2+pRESywCxShMThw4nZupW4777D1aVLhvegth44QNCoUYRVq0bQAw9gXbmSf24OnT+/yX/+k8TPP8exffsFXn7ZSbVqmVuQDGDfPivvvx9A+/ahVK8exlNPBbJsmQ2XK9NFiYiIiMjfsh2iT58+zYoVK/jzzz9Tnfvyyy9p2rQp5cuXp2/fvuzduze7txMR8S8WC+527XzbZf32G87nnsNbokSGHmq4XDhmzSK0WzdCGzTAMW4cxtmzKa4pXdrkyScTWbs2lmXLzjB48BFq1Mh8Cj52zMLnnwfQs2cIFSvmYfDgIObMsREfn+miRERERG5q2Q7Rn3zyCT179mTjxo0pjk+aNImnnnqKnTt3cu7cOX755RciIiI4+683iCIiNwqzeHESR4wgZts24qZPx9WxI2YGJyVb//yToBdfJOyWWwgaPBjr2rUpeqcBbrnFzaBBx/jll7Ns3XqBV19NoHFjN4aRuVk5588bzJzpYMCAECpUyMPddwczfbqdc+c0gVpERETkSrIdoletWoXVaiUiIiLF8XfffReAxx57jKlTp9K4cWNOnjzJxx9/nN1bioj4N5sNd6dOxM+YQcy2bTiffhpv0aIZeqiRmIhj5kxCO3cmtHFjHJ98AtHRqa4rW9bk0Ud9Q7737Inh/ffjad/ehd2euUCdkGAwb56doUODqVgxjO7dQ/j0UwdHjypQi4iIiKQl2yE6KiqKIkWKEBoamnzst99+Iyoqittuu43/+7//o0uXLnz55ZdYrVYWLlyY3VuKiOQaZunSJI4cScxvvxE3eTKuNm0y/Fjr7t0E/fe/5LnlFoIefhj75s2peqcBChc2ue8+F99+G8++fRf4/PN4evbM3LZZ4NuLesUKG888E0S1anlo2zaE994LYO9eLZ8hIiIiclHWl33929mzZ6lRo0aKY+vXrwegc+fOyceKFClC+fLlOXDgQHZvKSKS+9jtuLt1w92tG5b9+7F/9RWOqVOxnD59xYcaCQk4vv6a/F9/TWClStCyJUbJkpjFiuEtWhSzeHFfT3dYGHnyQO/eLnr3duF0JrBihY25c+3Mn2/jzJnMheHNm21s3mzjlVcCqVLl4tZZbmrX9mjrLBEREblpZTtEWywWYmNjUxyLjIzEMAwaNWqU4niePHkUokXkpuctV47EUaNIfP557HPn4vjyS2yrVmXoscF798JlFmk0w8J8ofrvcB1QvDgRRYvSpV0x3HcVY9OxksxaV5Kffg7h8OHMBeo9e6zs2WPlnXegZEkvXbr49qJu3NiDLdv/k4iIiIjkHtl+61O6dGn++usvzp07R758+XC5XCxdupSgoCDq1KmT4tozZ85QoECB7N5SROTG4HDg6tULV69eWP78E8ekSdinTcNy7lyWijNiYrDGxFw2ZLf9++uDQoWIq1iMKG8Jfj9bgh3RJTlKcY5QIvnPMxTAvMyMn8OHLUyYEMCECQHkz++lUyffXtStW7vR9pYiIiJyo8t2iG7Tpg179uxh4MCBDBo0iJ9++omzZ88SERGB7R/dE+fPn+fAgQPUq1cvu7cUEbnheCtWxPnqqzhHjsQ+e7avd3rduqtyL8upU4SdOkU1tlPtMtckYecoxZND9T8D9j//PHs2lGnTHEyb5iAkxKRdOzcRES5uv93FP5bKEBEREblhZDtEP/HEE/zwww8sW7aM5cuXY5omgYGBjBgxIsV1CxYswDRNGjdunN1biojcuAIDcfXrh6tfPyy7duGYNAnH9OkYFy5c02o4cFGWg5TlYLrXnSfPpVAdV5wjP5Vg3k8NGB7YhdYdLfTqlUSHDm6Cgq5RxUVERESusmyH6MKFC7N06VI++OAD/vzzT0qVKsWQIUOoUqVKiuvWrVtHjRo16NixY3ZvKSJyU/DecgvON97A+fLL2GfNwjp9OpbffsN+/vz1rlqyvFwgLxe4hd0pjh9xFuf9n4bx2E8P4Q3NQ6dOLnr1ctGmjZuAgOtUWREREZEcYERHR2duDxSRq8TpdBIVFUWpUqUI1MRKv6A28S/J7VGoEEHR0ViOH8dy7BjGsWO+P//9fXz89a4yFwhjIoMZyxMcoSR58ph07epbPbxFCzd2+/WuYfboNeJ/1Cb+R23iX9Qe/kdtkvtoTVURkdwmMBCzbFk8Zcviudw1pgkXLmA5fjw5VKcZuI8fx/BctpRsy0MMT/MOw3ifr7mLty88zddf1+Trrx3kz++le3cXPXu6aNrUg9V61aohIiIikmNyNESfPHmSZcuWsXfvXmJiYggLC6Ny5cq0bt2aQoUK5eStREQkPYYBefPizZsXqlS5fNj2eDBOn74UtI8fxzh6NEX4No4dw3L2bLaqY8fNfUzmPibzM7fzFs+w7GxrvvwygC+/DKBIEV+g7tXLRcOGHiyZ24FLRERE5JrJkRCdmJjIiy++yFdffYXL5Up13m6388ADD/DKK68QoMlwIiL+w2rFLFIEs0gRvLVrX/46p9PXe53WEPKjRy/1bGdgCHknFtCJBWymLm/xDN/RhxMnbEycGMDEiQGULOmlRw9foK5Tx4Nh5NzTFREREcmubIdor9fLnXfembwyd6FChahUqRJFixbl+PHj7N27l1OnTjFx4kT+/PNPvv32Wwy9IxIRyV0yOITctmQJAR98gG3lyisWWY8tfMOd7Oc53mU4X/Af4gnh8GELH34YwIcfBlC2rIdevXxDvmvU8CpQi4iIyHWX7QFzU6dOZdmyZYSFhfHBBx+wc+dO5s2bx+eff868efPYtWsX48aNI2/evCxdupRp06blRL1FRMTfGAbudu2Imz2bmOXLSerdGzMD47LLcYBxPM4hSvN/vEhhTiSfO3DAyrvvBtK8eRi33RbKmDEB7Nmjsd4iIiJy/WT7nciMGTMwDIPJkyczYMAAbLaUndtWq5V77rmHSZMmYZom06dPz+4tRUTEz3lr1ybh88+J2bKFxMGDMYODr/iYApzlRV7lIGX4hIeoxB8pzv/xh5U33gjkttvCaNo0lHfeCWD/fgVqERERubay/e5jx44dlClThpYtW6Z7XcuWLSlbtiw7duzI7i1FRCSXMMuWxfnmm8T8/jvOF17Am4FFJgNJ5CEmspuqfE8vGrEu1TU7dlgZPTqQOnXCaN06hHHjHBw+rLHeIiIicvVlO0QnJCSQL1++DF2bL18+nE5ndm8pIiK5jJk/P4nPPEPM9u3Ejx2Lp2LFKz7GgkkvZrGOJqyiGd34CQNvqut+/dXGiy8GUaNGHjp2DGHCBAfHjytQi4iIyNWR7RBdpEgR9u7dS0JCQrrXxcfHs3fvXgoXLpzdW4qISG4VFITr/vuJjYwkbupU3A0bZuhhzVjDT/RgJ9V4kE8JIO0PZDdssDFiRBC33BJG164hfPmlgzNnFKhFREQk52Q7RDdv3py4uDief/75dK97/vnniYuLo0WLFtm9pYiI5HYWC+6uXYlbtIjYBQtwde6MmYGlt6uyh08ZzAHK8jz/Ix9p719tmgarV9t48skgKlcOo3fvYKZOtRMdncPPQ0RERG462Q7Rw4YNw26389VXX9GsWTOmTp3Kli1bOHz4MFu2bGHq1Kk0bdqUyZMn43A4ePzxx3Oi3iIicoPwNGpE/NdfExsZSdJ992EGBFzxMUU5wf8YySFK8x5PUJqDly/fY7BkiZ1HHw2mUqU89O8fzOTJdk6cUA+1iIiIZF6294muXLkyn3zyCQ8//DA7duxIMySbpklgYCDjx4+ncuXK2b2liIjcgLyVKpHw/vs4n38ex8SJOD7/HMsVuo5DieMJ3udRPmQm/XiLZ9hKncte73IZLFxoZ+FCOwD16rm5/XY3HTu6qFlT+1CLiIjIleXI3iA9e/Zk5cqV3H333RQuXBjTNJO/ChcuzIABA1i5ciU9evTIiduJiMgNzCxShMQXXyTm999JGDMGb6lSV3yMDQ93MZ1fqcti2tGeRYB5xcdt3mzjf/8LpEWLMGrWDOOppwJZvNiG1sAUERGRy8l2T/RFlSpV4sMPPwTgwoULxMbGEhoaSp48eZKvadmyJefPn2fr1q05dVsREblRhYaSNHQoSYMGYf/xRwI++ADr9u1XfFg7ltCOJWzlVt7maWbQHzf2Kz7u8GELn38ewOefBxAcbNKqlZvbb3fRsaObIkWuHMhFRETk5pAjPdH/lidPHooXL54iQAMcPnyYQ4cOXY1biojIjcpmw9WnD7ErVhD744+42rTJ0MNqs42pDOCQvQJP8i6hxGT4lvHxBvPn23n88WCqVMlD27YhvPlmANu3WzCVp0VERG5qVyVEi4iI5DjDwNOqFfE//EDMqlUk9euHabvygKpirije5SlOBpTi0wLPUpwjZGSo9z9t3mzjtddSDvtetEjDvkVERG5GOTacW0RE5Frx1qxJwsSJOF96iYDx43F89RVGbGy6jwlKPM+DiW/xIG/hNSw4LcHEeIKJI4Q4Qogng38/HMK5z0P4/PMQJjqCKFfDS5P2h2hxezAFywRjBgdDQABapUxEROTGpBAtIiK5llmyJM7//Q/nM8/gmDSJgPHjsZw4ccXHWUwvwZ5Ygkk/eF9RErDl7683/lEviwWC/w7UQUGYISGX/h4cjBkS4jv/97mLfyckBDM8HHe9ephlymSvbiIiInJVKESLiEjuFx5O0hNPkDR0KPaZMwn48EOse/Zct+oYXi/Exl6xdzw9nipVcLdrh6tDBzyNGvl6t0VEROS605xoERG5cQQE4BowgNh164j75hvcTZpc7xplmXXPHgI++ojQ7t3JU6ECwXfdhX3SJIzDh6931URERG5q6okWEZEbj8WC+/bbcd9+O9ZNmwj44ANsc+Zg5NKltY3YWOzz52OfPx8AT7VquNu3x9Wuna+X2n7lLbxEREQkZ2Q6RL/xxhtXvugyEhISsvxYERGRrPDUr0/85MlY9u/HtngxxrFjGHFxGPHxkJCQ6u/Ex2PEx2MkJEBcnG9otp+x7tyJdedOAt5/HzNPHtytWuFq1w53+/aYxYpd7+qJiIjc0DIdol9//XWMLK44appmlh8rIiKSHd5y5UgaPDhzDzJNSEy8FKjj4y+F7Ph4XNHR7N9xnBP7gzm4w8Wxv5wEeuL/Xss7jmCu/HdLJrfb+jfjwgXss2djnz0bAE+NGrg6dMDdvj2eBg0gA9uAiYiISMZl+n/WJk2aKAiLiMjNwTAgMBAzMBDy5UsVdxOdTjy3RlGnVCkaBwYSGwvLl9tYsMDOokU2Tp680tIjJoE4k0N1KLE0YCOdmU8HFpGP6ExX2fr771h//x3efRczb15cbdrgbtcOd7t2mEWKZLo8ERERSSnTIXrevHlXox4iIiK5XmgodO3qpmtXN14v/PqrlZ9/9oXq33+3pvEIAydBOAniLAUA2EU1JnMfVtw0Yj2dmU8nfqYOWzNdH+P8eRyzZuGYNQsAd+3avkDdoQOeevXAmladREREJD0a4yUiInIVWCxQr56HevU8jByZSFSUwaJFdhYssLFypY3ExPRHdXmwsYZmrKEZL/AaxTjK7SxI7qXOQ0ym62TbuhXb1q3w9tt48+XD3bYt7vbtcbdti1mwYBafqYiIyM1FIVpEROQaKFXKZODAJAYOTEoe9r1woZ2FCzMy7BuOUZwv+Q9f8h9suGjC2uRe6lr8lun6WM6dw/Hddzi++w7TMPDUrXupl7pOHd+nACIiIpKKQrSIiMg1ltaw7wULfMO+f/vtykOs3dhZSUtW0pL/8gYliaITP9OJn2nHL4QRm6n6GKaJbfNmbJs3wxtv4C1YEHebNrg7dMDdpg1m/vxZfaoiIiI3HIVoERGR6+ifw75feCGRI0cMFi/29VKvWGEjPv7Ki3kephSfMphPGYydJJqxOrmXujo7M1+n06dxzJyJY+ZMTIsFT/36vn2p27fHW6uWeqlFROSmphAtIiLiR0qUMLn/fhf33+/C6YTVq20sXOgL1YcOXTm8unCwjDYsow3P8DZlOJDcS92WJYQQn6n6GF4vtshIbJGRBP7vf3gLF8bdsiWeevXw1K+Pp2ZNCAjI6tMVERHJdRSiRURE/FRgILRr56ZdOzdvvulkzx5LcqDesMGKx3PlXuqDlOUThvIJQwnASQtW0omf6cx8qvBHputkOXkSx7ffwrffAmA6HHhq1boUquvXx1u2rG97MBERkRuQQrSIiEguYBhQtaqXqlWTGDYsiehoWLLEtzDZL7/YOHv2yr3UiQSymA4spgPDeY/y7LvUS20sJdB0Zr5eSUnYNm3CtmkTTJgAgLdAAV+o/jtYu+vVg/DwTJctIiLijxSiRUREcqHwcOjd20Xv3i48Hti0ycqiRb5e6rT3pE7tLyrwEY/yEY8SaCbQiuV04me6GPOpYO7Lct0sZ85gWbQI+6JFycc8FSum6K32VK8ODkeW7yEiInK9KESLiIjkclYr3Habh9tu8/Dii4kcPmywePGlPakTEq48tNpJEAvoxAI6Mcz8gIrspTPz6cx8WhkrCDATs1fHP//E+uefMGMGAGZAAJ5bb03RW22WKaNh4CIi4vcUokVERG4wJUuaPPBAEg88kERCQsrFyaKiMray9p9U4gOG8QHDCDbjaM0yOjOfNtaVVPbsxIKZrToaiYnJC5Zd5C1UCE/dusm91e66dSFv3mzdR0REJKcpRIuIiNzAgoKgfXs37du7eestJ7t3p1yczOu9cs9vPCHMoyvz6AoeCOMC9dnEbWyghWM9txmR5E88nu26Wk6dwrJwIfaFC5OPeSpXTtFb7a1eHez2bN9LREQkqxSiRUREbhKGAbfc4uWWW5J44okkzp0zWLLExqJFNhYvtnHuXMZ6qWPIk7yN1utJACaliEoO1a2DN1Aldgt2d0K262z94w+sf/wB06cDYAYF+YaB16+Pu359PPXqYZYsqWHgIiJyzShEi4iI3KTy5TPp08dFnz6+xck2bvQtTrZggZ2dOzO2OJmPQRSliaI03yX1hSSw4aIGv9Pcvp7b86+nnjuSImd2Z7vORkICtvXrsa1fz8Xdqb1FilxatKxWLaz58oGZveHmIiIil6MQfQOYPXs2X3zxBVu3biU6Oppt27ZRpkyZ610tERHJRaxWaNTIQ6NGHl56KZGoKINFi+wsWmRjxQobTmfmenrd2NlKHba66jDuxFAA8hJNI0skPYqtp5l9AxXPRhJ44XS26245cQLL/PnY588nEMgPeENCMEuUwFuihO/P4sXxliyJWbw43r+/13xrERHJCoXoG0B8fDxNmjShc+fOPPvss9e7OiIicgMoVcpk4MAkBg5MIj4eVq3yDfteudLG3r2Z6aW+5DzhLPR2YOGRDn8fMSnHfnqXXE/H8PXUSoikYNRWLEnZWwkcwBIXBxeHgl+GGRaWHKiTg/bF0H0xaOfJk+26iIjIjUUh+gZwxx13ALBz587rXBMREbkRBQdDx45uOnZ0A3DypMG6dVbWrrWxdq2N33+3YJpZmZNssJ/yvH24PG8fvgsAO0l0LvErPYuvp5GxgTInNhJ48M8cfDb/uHtMDNbdu7HuvvwwczNPnisH7bCwq1I/ERHxT7kiRB89epQff/yRxYsXs3fvXk6cOEG+fPm47bbbGDZsGPXr17/mdZoxYwbr1q1j69at7Ny5k6SkJD766CPuvvvuyz5my5YtjBkzhg0bNuB2u6lWrRqPPPIIPXv2vIY1FxERyZ7ChU26d3fTvbsvVEdHQ2SkjbVrraxbZ2PLFisuV9YW+nLh4Kcjt/HTkduSj1Urepo7K66nTcgGbomJJO/uTVjOns2Jp3JFxoULWC9cwLpr12WvMfPkwVuy5OWDdokSEBJyTeorIiJXX64I0RMnTmTs2LGUK1eO1q1bU7BgQfbt28e8efOYN28en332Gb169bqmdXr11VeJioqiQIECFClShKioqHSvX7lyJb179yYwMJBevXoRGhrK7NmzeeCBBzh8+DCPPfbYNaq5iIhIzgoPhw4d3HTo4AYSiY+HzZsv9VRv3GglPj7rq2fvPF6QF4935UW6ApA/n4ceLfcSUXg99TyRFD24Edtv2zGSknLmCWWSceEC1p07saYzIszMmzc5UJvFiuEtUAAzf37MfPl8f+bPj3nxWHi4b5K6iIj4pVwRouvWrcvcuXNp1qxZiuNr166le/fuDB8+nC5duhAQEHCZEny++eYbmjRpQunSpdM87/F4GD9+PIMHD8bhcKRb1rhx4yhfvjylS5fmvffe45VXXrnstW63m2HDhmGxWJg3bx61atUC4Nlnn6Vt27aMHj2a7t27p6jXqFGjGDt2bLp1iI6OTve8iIjI9RAcDM2be2je3AMk4nLBtm1W1q71Bet162ycP5/1UH32nJUvVlTlC6oC9xMaatK0SRzdy/5K44ANFIz9nQIJMdiPH8dy5AjG0aMYLleOPb+sMM6fx3r+fLpB+yLTMDDz5r0Urv8dtv/+8l48/nf4JijoGjwTERHJFSG6W7duaR5v0qQJzZs3Z+nSpezcuZM6depctowjR44wbNgwihQpwty5c1MFaa/Xy9ChQ5k5cyYOh4PBgwenW6dWrVpluP4rV65k//793H333ckBGiBv3rwMHz6chx9+mOnTpzNixIjkc48++ih33XVXhu8hIiLir+x2qF/fQ/36Hh5/PAmvF3btsvzdU+0L1idOZGyP6rTExhosXB7KQpoDzQkI8NKqVRI9Bnjp1MlFeB4vxunTvkB95AiWI0ewHD166e9HjmAcO3bdg/ZFhmliREf7xsn/9VeGH2cGBaUI3N4rhfACBXwLp1my/rMXEbkZ5YoQnR673Q6A9QrDnkqUKMFnn33GAw88QEREBHPnzqVUqVKAL0APGTKEmTNncscdd/Dggw/maB1Xr14NQJs2bVKda9u2LQBr1qxJcbxgwYIULFgwR+shIiLiDywWqF7dS/XqSQwa5NvSef9+C2vW+OZUr11r5cCBrA9nTky0sHBhIAsXgs1m0rKlm27dAujSpQgFL/eBu9eLceqUL1wfPpwyaB89iuXwYV/QdruzXK+rzUhIwDhyBI4cyfBjTIslZcD+u2fbU6MG7o4d8ZYvfxVrLCKSO+XqEB0VFcXy5cspWrQo1atXv+L1ERERfP755wwcOJCuXbsyd+5cSpQokdwD3a9fPz7++GMsOfyJ7L59+wCoUKFCqnNFihQhNDSUvzLxSfO/nTt3jqioKPbv3w/Anj17OH/+PKVKlSJfvnxZLldERORaMAwoX95L+fJeBgzw9QYfPWokB+p162zs3Jm1UO12GyxZYmfJEjtPPmnSpImHbt1cdO3qonhx89KFFgtmkSJ4ihSB9IL2yZOXgvbRo8nDxS1HjviC9vHjfh20/83wejHOnIEzZ1KffO45PJUq4e7QAVfHjngaN/YNKxARucnl2hDtcrl46KGHSExMZNSoUVfsib6oe/fueDweBg0aREREBHXq1GHWrFn07t2b8ePH53iABrhw4QIAeS6z12RYWFjyNVkxf/58HnnkkeTv+/XrB3DF1cJFRET8VfHiJr17u+jd2xeqz571bat1MVhv22bF48ncvGqv12D1ahurV9t49tkgGjRw062bi4gIF2XLmlcuwGLBLFoUT9GiULdu2td4POkH7SNHME6evG6LoGWWde9erHv3EvDRR5h58uBq2xZ3hw6427fH1Ig5EblJ5coQ7fV6efjhh1m7di333Xdf8j7JGdWrVy/cbjeDBw/mwIEDdOnShYkTJ2Y4iPubu+++W2FZRERuaPnzm3Tp4qZLF18vb2wsbNxoSx4CvmmTlcTEzIXqjRttbNxo48UXg6hVy9dD3a2bi8qVvVmvqNWKWawYnmLFoF69tK8xTYiLwzh7FuPsWSznziX/3Th7FuPMGYx/HLOcPev7PhsfuOcE48IFHLNm4Zg1C9Mw8DRokNxL7a1RwzekQETkJpDrQrTX6+WRRx7h22+/pV+/frz33nuZLsM0TVauXJn8/e7duzl58iTFihXLyaomu9gDfbne5piYGMLDw6/KvUVERG5EoaHQurWb1q1922olJsKWLVYWLYKffrLy11+ZW6l6+3Yr27dbefXVQKpW9RAR4euhrlnTm/PZ0DAgNBQzNBSzdGkyHNldrhTh+p9fln9+/+9rPJ4cfgK+xc9skZHYIiMJfPVVvCVK4OrQAXfHjrhbtPAt0S4icoPKVSH6Yg/0N998Q58+fbI0/No0TYYNG8bUqVPp1asXt99+O0OHDiUiIoI5c+ZclSB9cS70vn37qF27dopzJ06cIDY2lrqXGxYmIiIiVxQQAI0be6hTx8ldd0WRmFiWhQtDmT3bzrZtmRtptnu3ld27rbz1ViDlynmIiPAN+65Xz3N9O1vtdszChTELF874Y0wTLlzAOHcOy5kzqQP4v3u8z57FOHIEw8zA8PZ/sBw5QsCXXxLw5ZeYgYG4W7b09VJ36ACFCmXyiYqI+LdcE6L/GaB79erFhAkTMj382jRNnnjiCSZPnkzPnj359NNPsVqtWCwWHnrooeRVu4sWLZqjdW/atCnvvvsuS5cupXfv3inOLVmyJPkaERERyRkVK3qoUSORp55K5MABgzlz7MyZYycyMnNvffbvt/LBB1Y++CCAEiW8dO3qG/LdqJGHXDELzDAgb17MvHnxlC2bsYecOYPtl1+wLVyI/ZdfMj2M3HA6sS9ciH3hQoKAoGrV8DZsiL13b2jShNzxgxMRubxcsTHgxSHc33zzDT169MjS/GXTNBk+fDhfffVVigAN0KdPHyZMmMD+/fuJiIjgxIkTOVr/li1bUrZsWb777ju2b9+efPz8+fO8++67OByOTM/rFhERkYwpW9bksceSWLQojp07L/DWWwk0b+7GYslcb+uRIxYmTAigS5dQqlYN48knA1m61IafbC+dY8wCBXD170/CF19wYd8+YufMIfGxx/BUrpyl8uw7d1Js0iTyR0QQVqkSQQ89hP2HH3z7YIuI5EJGdHR05v4HuQ7GjBnDG2+8QWhoKEOGDEkzQHfp0oVatWpdtoyjR4/SsmVLmjZtymeffYbNlvqT6G+//ZYhQ4bw8ccf079//3TrNHnyZNatWwfAzp072bZtG40aNaJcuXIANG7cmHvvvTf5+pUrV9K7d28CAwPp1asXoaGhzJ49m6ioKEaPHs1jjz2WoZ/FjczpdBIVFUWpUqUIDAy83tUR1Cb+Ru3hf9Qm/iczbXL6tMH8+TZmz7azYoUNlytrY7XDw7106uQb8t26tZsb+Z+CZf9+bAsX+r5Wr8bIxicIptWKp3FjXB07+vakrlRJi5NdA/q95X/UJrlPrgjRQ4cOZfr06elek5HtnA4cOEDJkiXTDNAX/fnnn1SsWDHbdbrzzjsZP358imObN29mzJgxREZG4nK5qFatGo888gi9evW64v1uBvoF4n/UJv5F7eF/1Cb+J6ttEh0NCxbYmT3bztKlNpzOrIW50FCTjh19Q77btXMTEpKlYnKHmBhsy5djX7gQ26JFWE6ezFZxnnLlfAuTdeyIu0kT30R3yXH6veV/1Ca5T64I0XJz0C8Q/6M28S9qD/+jNvE/OdEmsbHwyy++HupFi+zExmYtUAcFmbRt6yYiwsXtt7vImzdLxeQOXi/WbduwLVjg66XeujVbxZmhobhbt/at+N2hA2aRIjlTT9HvLT+kNsl9cs3CYiIiIiLXQmgo9OjhpkcPN05nAkuX+gL1zz/bOX8+44E6IcFg7lw7c+fasdtNWrXyBeouXdwUKHCD9WFYLHjq1MFTpw6Jzz2Hcfw4tkWLsC9ciHXZMizx8ZkqzoiNxT5nDvY5cwBw162Lu2NH357UtWpBJndnERHJSeqJFr+hT+H8j9rEv6g9/I/axP9czTZJSoJVq3yBet48G6dPZy3IWSwmNWt6qV/fTb16HurX91CxoveGzYXO8+c5P3s2JbduJfCXX7AcPJit8rxFi+Ju3RpPzZp4qlfHW706ZsGCOVTbG59+b/kftUnuoxAtfkO/QPyP2sS/qD38j9rE/1yrNvF4YN06K7Nn+3qajx7NXgLOm9ekXr1Lobp+fc8N01udok0CArD88Ydv+6wFC7Bu2IDh8WT7Ht7ChfFUq4a3WjU81av7wnWVKhAUlAPP4Mai31v+R22S+2g4t4iIiEgmWa3QrJmHZs08vP66k82bfYF69mw7Bw9mPlCfP2+wdKmdpUvtycfKlfOF6Xr1PDRo4KFGDU/uX2vLMPBWqUJSlSokPf44REdjX7LEN4968WIs585lqVjLyZO+hc2WL08+ZloseMuXvxSsq1XDW7063rJlNRxcRLJFIVpEREQkGywWaNDAF3T/7/+c/Pabhdmz7cyZY2fPntTbcmbU/v1W9u+38u23vu8dDpNatS6F6vr13ZQpY+buXaHCw3H17o2rd2/weLBu3OjrpV64EOvOndkq2vB6sf75J9Y//8Q+e3bycTM4GE/Vqnj/DtYXw7WGhItIRilEi4iIiOQQw4BatbzUqpXIyJGJ7NljSe6h/u23rAdqgKQkg02bbGzaZGPCBN+xggW9KYaA16njJjw8+8/jurBa8TRqhKdRIxJffhnj0CHsixZhW7QI24oVGImJOXIbIz4e25YtsGVLiuPewoV9w8D/DtYaEi4il6MQLSIiInKVVKni5ZlnEnnmmUT277fw8882IiOtbNpk4/Dh7A8pPn3awsKFFhYuvDQMvHLlS6G6Xj031at7seXCd3xm6dIkPfggSQ8+CHFx2FauTA7Ulv37McycnTOePCR82bJLdbBY8FaocClYa0i4iKAQLSIiInJNlCvn5eGHk3j4Yd/3x48bbNpk/fvLxq+/WomLy/7Y7D/+sPLHH1a+/tr3fVCQSe3aF4O1b/GyEiVy2TDwkBDcnTrh7tTJ931cHNbdu7Hs2IF1506sO3di2bEDy5kzOXpbw+vFuncv1r17sf/0U/JxMyTENyT8X+FaQ8JFbg4K0SIiIiLXQdGiJl27uuna1Q0k4vHA7t2W5FC9ebOVXbssmGb20m5CgsG6dTbWrbMBvpXJihW7OAzcF6rr1PEQGpr953TNhITgqVcPT716uC4eM02MkyeTA7V1504sO3di3b0bw+nM0dsbcXHYNm+GzZtTHPcWKeIL1JUqYebJA8HBmH9/ERJy6e+hoamOExRE7vpkQ+TmpRAtIiIi4gesVqhe3Uv16l7uu88XDS9cgF9/tbJ5sy251/rkyewPIz52zMLcuRbmzvUNA7dYTG65xZs8BLxJEw8VKnhzV6YzDMwiRXAXKQKtW1867vFg+esvX6D+R7i+KkPCT5zAcuJEiiHhGWUaRnLoJjgY82K4vvj3kJBLoTy9v/8zrP8zoGv4uUiOUYgWERER8VN58kDLlh5atvTtpWyaEBVlpAjV27ZZcTqzl3a9XoMdO6zs2GHlq68cAJQu7aVNGxdt2rhp2dJN3rzZfjrXh9WKt1IlvJUq4e7e/dLxazQkPKMM04S4OIy4uKtS/sVgHRoURFhgIJbq1TGbNcPdpAneqlUVsm8kpgnx8RAScr1rcsNSiBYRERHJJQwDSpc2KV3aRc+evt5qlwt27LCwcaMvWG/ebOXPP7O3EjjAoUMWJk0KYNKkAKxWkwYNPLRp46ZtWze1a3uwZv8W19d1HhJ+rRnx8Rjx8QAEA+zdCz/+CIA3PBxPo0a4mzTB06QJnltvBbv9smWJH/J4sK5ejeP777HNnYvl7Fk81aqROGwYrj59yP0vWP9iREdH5+w4FpEscjqdREVFUapUKQIDA693dQS1ib9Re/gftYn/UZv4nDtnsHmzNTlUb9pk5dy5nOtpzJ/fS6tW7uRQXazY5d9O3hBtcg2HhPsDMzgYT/36uBs39gXrBg0gOPh6V+uGleXXiNeLNTIS+/ffY//pJ9/q8mnwVK5M4ogRuHr0UJjOIQrR4jduiP9kbzBqE/+i9vA/ahP/ozZJm2nCX39Z/rEauJXff7ficuXMpOdq1S71Ujdu7OafP/obuk3SGhK+cyeW06evd81ylGmz4aldG8/FUN2oEWa+fNe7WjeMTL1GTBPLtm04vv8e+6xZWA4fzvB9PLfcgvO//8UdEaHh+9mkEC1+44b+TzaXUpv4F7WH/1Gb+B+1ScY5nbB9uzXFNluHDmX/jXVQkEnTppd6qUuXjufw4ZusTbxeiI/HiIvzDaG++Oc/j/3777GxaR//+/vkv/89JPt681Sr5gvUjRvjbtwYs3jx612lXCsjv7csu3b5epx/+AHrX39l636e6tV9YbprV60In0WaEy0iIiJyEwoMhIYNPTRs6Ek+dvKkwapVNpYssbFsmY1jxzIfqhMSDH75xc4vv/jm1JYoEUz9+ha6dLHToQOEh+fUM/BjFotvG6vQUHK8t8rrhYSES8E6kwHdjInBs28fwfv2ZasaF3ve+ewzX7XKlMHdpAnuxo3xNGmCt0IFBbRssvz116XgvGtXjpVr3bGDkAED8NSqhfO553DffrvaKpMUokVEREQEgMKFTXr3dtG7twvThJ07LSxdamPpUhtr19pITMz8G+0jR6wcOVKIn37ybaVVr96lod9163qw6d1o5lgsvq2r/l55ObMh/WKvZ+mQEEK2bsW2bh3Wdeuwbt2K4XZnvVoHD+I4eBDH9OkAeAsXTu6ldjdujLdGDc3HzQAjKgr7jz9i//57bFu3ZqkMb8GCmKGhWA8cSPc66/bthNx5J+46dUh8/nnc7dopTGeQhnOL39AQPP+jNvEvag//ozbxP2qTqyc+Htau9fVSL11qY8+e7AeivHlNWrVy07atbyutkiX1tvRqu+xrJC4O66ZN2Nau9QXrjRsxEhJy7L5mnjy4b7stOVh76taFgIAcKz83S4qKIn7yZIqtWIEjMjJLZZh58+KKiMDVuzfu5s0BsH/3HQFvvIF1//4MleFu0MAXplu1Upi+AoVo8Rt64+N/1Cb+Re3hf9Qm/kdtcu1ERRksW2ZjyRI7y5fbOH8++2+6q1S51EvdpIlbC0JfBRl+jSQlYd22Deu6dcnB2jh/PsfqYQYE4KlX79K86oYNISwsx8r3d8a5c9jmzMHx/fdYV63C8HozXYYZEoKrc2dcvXrhbtMm7Q8l3G7s33xD4JtvYjl0KEPluhs3xvncc3hatMh0nW4WCtHiN/TGx/+oTfyL2sP/qE38j9rk+nC7YcsWa3Iv9ebNVrze7IXqgACTxo19gbpNGzfVqnnVOZYDsrOdkmXXruTh37a1a7EcO5Zj9TItFrw1a16aV123LmaxYjfWEPCYGOzz52P/4QdsS5Zkafi8GRCAu0MHknr3xt2hQ8a3HktKwj59OoFvvZXhFb3dzZrhfP55PE2aZLqeNzqFaPEbeuPjf9Qm/kXt4X/UJv5HbeIfoqNhxQpfL/WSJVaOHMl+ECpWzEvr1r5Q3aqVmwIF9BY2K3LsNWKaGAcPXhr+vXYt1mwuVpbqFnY73hIlMEuVwluqFN7SpX1//v13s0QJsNtz9J45LiEB26JFOL7/HtuiRRhOZ6aLMG023G3a4OrVC1fnzpAnT9brk5iIY+pUAt55B8vRoxl6iKtVKxKfew7Pbbdl/b43GIVo8Rt64+N/1Cb+Re3hf9Qm/kdt4n8SEpysWnWK3bvLsHJlIGvW2EhIyH6XcrVqHpo0cdOkiYfGjd0UK6a3tBlxNV8jxokTWNev9wXrtWux/P47hnn12sW0WDCLFbsUrv/+07z495Il4Xr8HkhKwrZ0KfYffsA+fz5GbGymizAtFjzNmvl6nCMiMPPnz9k6Op04vvqKgPfew3L8eIYe4mrXzhem69XL2brkQgrR4jf0xsf/qE38i9rD/6hN/I/axP/8u02cTli37tICZTt35sxw3fLlPTRu7AvWTZu6KVPG1PDvNFzT18j589giI5OHf1u3bMFISrq69/wXb+HCqcP1xd7sUqVybh6224119Woc33+PffbsLM8fdzdq5Otx7t4ds0iRnKlbehIScHz5pS9MnzqVoYe4OnbE+dxzeGvXvrp182MK0eI39MbH/6hN/Ivaw/+oTfyP2sT/XKlNjh41krfRWrbMxrlzmd+bOi3Fi3tp3NjXU92kiZsqVbxYcqboXO26vkYSErBu2eIL1OvWYYuMzFIvbU7y5suXOlxfDN2lS2OGh19+pWqvF+uGDb4e5x9/zHAI/TdXrVocb9WKwHvvxVGxYtafTHbExeH44gsCxo7FcuZMhh7i6twZ53//i7dWratcOf+jEC1+Q298/I/axL+oPfyP2sT/qE38T2baxOOBbdsuLVAWGWnF48mZ7uT8+b00buwb+t20qYeaNW/OPar96jXidmP5/Xdsa9YkL1iW0QB3rZhhYanDdYkSWH/9FfusWViOHMlSuZ6qVXH17o2rVy/iS5TwnzaJjSXg009xfPABlnPnMvQQV7duvjBdrdpVrpz/UIgWv+FXv9QFUJv4G7WH/1Gb+B+1if/JTpucPw8rV/oC9ZIldg4dyrmu5NBQk9tuuzSnum5dz3WZPnut+fVrxDSx7N2LdfNmLIcOYYmKwhIVhXHoEJbDhzFcrutdw2zxlCuXHJz/GTj9sk0uXCBgwgQCPvwwQ0PTTcPA1bMniSNG4K1S5RpU8Pq6CT9/ExEREZHcIG9eiIhwExHhxjSd7NtnYflyG+vWWVm71saxY1kP1bGxxt+rh/tWdw4IMKlX7+Kcag8NGrgJDc2pZyIZYhh4K1fGW7ly6nNeL8aJE6nDdVRU8jEjIeHa1/kKvCVK4OrZE1fv3nhq17780HB/kycPic88Q+KgQQSMH0/A+PEYFy5c9nLDNHH88AP2WbNw9e1L4rPP4r1eQ9OvAfVEi9/wy0/hbnJqE/+i9vA/ahP/ozbxP1erTUwTDhywsHatL1CvXWtl//6c21PYajW59VZP8pzqxo095MuX+98237CvEdPEOHMGy6FDGBeD9T8CtyUqKt0QmJO8hQrh6tEDV69evm2hrjAZPze0iXHuHI6PPiLgk08yNI/dtFhw9e/vC9Plyl2DGl5b6okWERERkVzHMKBcOS/lynm5+27fMN9jxwzWrbMlB+vsrPzt8Rhs2WJjyxYbH34YAGhbLb9mGJgFC+IpWBDq1k37mujoFD3XF/+eHLrPns3y7b3h4bgjIkjq3RtPs2bcaBPuzXz5SBw5kqShQ3F8+CEBEyZgxMdf9nrD68UxfTr2mTNx3XUXzqefxixT5hrW+Oq6sVpXRERERG5axYqZ9OrlolcvX6g+e9ZIHvq9bp2Vbduyt1DZzp1Wdu608tlnvu/LlbvUU61ttXKB8HC84eF4a9ZM+3xsLJbDhy8ND/934P7XfspmaCiuzp1x9e6Nu3VrcDiuwZO4vswCBUh8+WWSHn6YgA8+wPHZZ+kOozc8HhxTpmCfPp2kAQNIHD4cs1Spa1jjq0MhWkRERERuSPnzm3Tp4qZLFzcAMTGwceOlnurNm60kJmY99e7f7xtCPm2aLzxpW61cLjQUb9WqeKtWTfu804nlyBGMw4chJARP9eoQFHRt6+gnzEKFcI4eTeKjjxIwdiyOL77ASEy87PWG203Al1/imDaNpHvv9YXp4sWvYY1zlkK0iIiIiNwUwsKgTRs3bdq4gUScTtiy5dKc6shIG7GxWQ/VR49a+P57B99/7/s+X76U22rVqnVzbqt1wwgMxFuhAlSocL1r4jfMIkVwjhlD4uOPE/Duuzi++gojKemy1xtJSQR89hmOKVNIfOQREl966RrWNufoZSwiIiIiN6XAQP7uNfYA4HbDb79ZWbPm0hDwc+ey3pV87pyF+fMtzJ/vWwE8NNSkQYNLc6rr1fPcrB2ZcoMxixXD+dZbJA4b5gvTU6akuyWZkZiI4fFcwxrmLIVoERERERF8a0HVqeOhTh0Pjz6ahNcLe/ZYknuqc2JbrWXL7Cxb5gvVDodvW62LQ8AbNnSTJ09OPRuRa88sWRLnu++S+MQTBL7zDvZp0zDc7tTXhYaS+Pjj16GGOUMhWkREREQkDRYL3HKLl1tuSWLgwJzfVispybea+Lp1Nt59FywWk5o1vX9vqeUL1gULagVwyX3M0qVJeP99nE8+SeBbb2H/5psUPc+JDz2EWaDAdaxh9ihEi4iIiIhkwNXeVsvrNdi2zbeK+Pjxvm21qlS51FPduLGbUqUUqiX3MMuWJeGjj0gcPpyAN9/E/u23EBJC0qOPXu+qZYtCtIiIiIhIFl1uW62LwTq722rt2WNlzx4rkyb5vi9Vyvv3XtW+YF2xolfbaonf81aoQMKECSQ+9RSWXbsw8+W73lXKFoVoEREREZEc8u9ttWJjfdtqrVnjC9abNmVvW62oKAszZjiYMcO3rVahQr4VwC8OAa9Rw4s1653hIleVt3JlvJUrX+9qZJtCtIiIiIjIVRIaCq1bu2nd2retVmIi/PrrpTnVGzbYiInJeqg+dcrC7NkWZs/2LVaWJ4/Jbbdd2qu6Th0PDkcOPRkRARSiRURERESumYAAaNTIQ6NGHoYPB48HfvvN8vfwb1+wPnMm6yuAX7hgsHixncWLfaE6MNCkfv2Le1W7qVkzp56JyM1LIVpERERE5DqxWqF2bS+1aycxdGgSpgl79/5zBXAbhw9nPVQ7nQarV9tYvdrGW2+BzRZClSohNG1q0LChQb16HsqW1bxqkcxQiBYRERER8ROGAZUre6lc2cv99/sWKzt0yEjRU713b9YnPbvdBjt2hLJjB0yc6DuWL5+XunU91K3roV4935+FC2sVcJHLUYgWEREREfFjpUublC7ton9/X6g+efKfK4Db+P13C15v1ruSz52zsGSJhSVL7MnHSpb0Uq+eh3r13NSt6+HWWz2EhWX7qYjcEBSiRURERERykcKFTbp3d9O9u28F8PPnITLS10u9bp2NzZutuFzZG599+LCFw4ct/PSTL1gbhknVqt5/9Fa7qVbNq0XL5KakEC0iIiIikovlzQvt27tp3963AnhCAmzefGkF8I0bbcTFZS9Um6bBrl1Wdu2yMm2a71hAgEmtWp4UQ8HLl/diyfoUbpFcQSFaREREROQGEhQEzZp5aNbMA4DLBdu3W1m71srq1QaRkVbOnbNfoZQrS0w02LjRxsaNlyJF3rwmdeu6UwTrokU1v1puLArRIiIiIiI3MLudv+c3exg0yMmhQ1EYRhl27Ahm82YbW7ZY2brVmu3eaoDz5w2WLbOzbNmlkF6ihJc6dS4NA69d20PevNm+lch1oxAtIiIiInITMQwoVcpLpUpuevTwzav2eGDPHgtbtljZssXK5s02duyw4HZnP1gfOWLhyBELc+deml9dqdKl+dX16nmoXt1DQEC2byVyTShEi4iIiIjc5KxWqFbNS7VqXu65x7cKeEIC/Pab9R/B2sq+fVnfXusi0zT44w8rf/xh5ZtvfMccDpMaNS5tsdWggYcKFbR/tfgnhWgREREREUklKAgaNvTQsKEn+Vh0NPz6q28F8M2bfeH6xInsrySWlGSwZYuNLVsuxZNixbw0b+6mRQvfV+nSmlst/kEhWkREREREMiQ8HFq3dtO6tW8YuGnC0aMGmzdb+fVX3zDwX3+1EhOT/S7kY8cszJzpYOZM3z5aZcp4kwN18+ZuLVgm141CtIiIiIiIZIlhQIkSJiVKuOnWzbfFltcLe/daUgwD/+237O9dffCghSlTHEyZ4gvVVap4kgN1s2Ye8udXqJZrQyFaRERERERyjMUCVap4qVLFy513+uZXJybCjh3WFMPA//gje/Or9+yxsmePlU8/DcAwTGrWvNRT3bixm7CwnHg2IqkpRIuIiIiIyFUVEEDy3tGDBvmOnT8PW7da2bLFljwc/MiRrM2vNk2D7dutbN9u5cMPA7BaTerW9SSH6oYNPQQF5eATkpuaQrSIiIiIiFxzefNCy5YeWra8tHDZsWMGkZFWVq2ysXKlLcu91R6PwcaNNjZutPHOO77Vvxs2vBSq69b14HDk1DORm41CtIiIiIiI+IVixUy6d3fTvbtv4bJjx4zkQL1ypY1Dh7LWU52UZLB6tY3Vq2289hqEhJg0bnxp5e+aNb1Ys797l9wkFKJFRERERMQvFStm0q+fi379fHOrDxzwheqLwfr48ayF6rg4g19+sfPLL3YA8uY1adr0Uqi+5RbtUS2XpxAtIiIiIiK5QtmyJmXLuhgwwIVp+lYBv9hLvWqVlXPnshaqz583mD/fzvz5vlBdqNA/96j2UK6cQrVcohAtIiIiIiK5jmFA5cpeKldO4sEHk/B64fffLX8Hahtr1tiIjc1a8j11ysIPPzj44QffxOmSJS+F6ubN3ZQsqe20bmYK0SIiIiIikutZLFCrlpdatZJ49NEkXC7f6t8Xe6o3bLDidGYtVB8+bGH6dAfTp/tCdfnyHlq2dNOqlZvmzbVH9c1GIVpERERERG44djs0aOChQQMPTz2ViNMJmzZZk3uqN2604nZnLVT/9ZeVv/6y8uWXvj2qa9Xy0qqVm5Yt3TRq5CY4OIefjPgVhWgREREREbnhBQZCs2YemjXzAInExsKGDRdX/raybZsVrzfzodo0DbZt8z3+/fcDkrfTuhiq69TxYFPquqGoOUVERERE5KYTGgpt27pp29a3nVZ0NKxZY0vuqd65M2t7Xv1zO61XX4U8eXwrf18M1VWqaJGy3E4hWkREREREbnrh4dCli5suXXyh+tQpXxheudI3BHzfvqyF6gsXDH7+2c7PP/tW/i5a1EvLlu7krwIFcuoZyLWiEC0iIiIiIvIvhQqZ9OzpomdP3x7Vhw/79qhescLXW330aNa20zp+3MKMGQ5mzPAtUlaxYjB16hh06uSgTRtfmBf/phAtIiIiIiJyBSVLmtx5p4s77/TtUf3nnxaWL78Uqi9cyNoY7T//tPHnn4X59luwWExq1740n/q22zwEBubwE5FsU4gWERERERHJBMOASpW8VKqUxKBBSXg8vu20VqywsXy5bzutxMTMh2qv12DLFhtbtth49134//buPKyKcvED+PdwwCVAQBBIdlATBBfMBbgguOQTpCLaQly3Jw1JvXo189Yt0+JetWx5rqmpN80F1MgQSa9JJwUuEKWB4D3CAUQUTDONVVnO8vuDH4dObHPgINv38zw8j7wz8847jO/M+Z6ZeWfAABUmTaoP1f7+cowerYC4fXeVkw4xRBMREREREXWAWAyMH6/A+PEKrF1bg4cPgfT0xlCdmSmGSqV9qK6uFiExsf5q9+bNgImJCn5+cvU7ql1cOEhZV2CIJiIiIiIi0qGBAwF/fwX8/RV4++0a/PabCMnJYnUgzs9v3+XksjIR4uMNEB9fP0iZjY3mIGXW1ipdbga1gCGaiIiIiIioE5mZqTB7thyzZ9eP/F1c3HiF+cIFMX75pX2huqRED9HR/RAdXT9I2ciRCnWg9vJSwMyMobozMEQTERERERE9Qra2KoSF1SEsrA4PH1YjMfEu8vLskZIyECkp+qioaN892jk5YuTkiLFnT38AgIODEuPGyTFunAJjxyowZoyCo3/rAEM0ERERERFRFxGJABeXavj7P8SqVSrU1QEZGWL1yN8//CBGXV37QnVRkR6Kivrh5MnGMmdnhTpUNwTrQYN0sy19BUM0ERERERFRN2FgAEycqMDEiQq89loNqqqA77/XV4fqrKyODc997ZoY166JceJEY9nw4Y2hetw4BUaPVsDIqIMb0osxRBMREREREXVThobAtGlyTJtW/zz1vXv1g5RduFAfrK9f7/g7r/LyxMjLEyMmpv53kUiFESOU6lA9bpwCHh4KPPZYh1fVKzBEExERERER9RDm5ioEB8sRHFwfqq9fFyEpSV89UNmvv+p1eB0qlQi5uWLk5opx/Hh9mZ6eCiNHKjWuWLu7KzBwYIdX1+MwRBMREREREfVQjo4qODrWYeHCOiiVgFSqh4sXxcjI0EdGhhhSqR7k8o6/TFqpFEEqFUMqFSM6ur5MLFbB1VXzivWoUQr079/h1XVrDNFERERERES9gJ4e4O6uhLu7EosX1wEAqqsBqVSMjIz6n8xMMa5e1YNC0fFgrVCIcOWKGFeuiHHkSH2ZgUF9sK4P1XKMHauAm5sS/fp1eHXdBkM0ERERERFRLzVgAODpqYCnp0Jd9vAhcOVKY6jOzBQjJ0cPSmXHg3VdnQhZWWJkZYlx8GB9cu7XT4VRozRHBXdzU0K/h6bRHtpsIiIiIiIiao+BA4EJExSYMKExWFdVNQbrhnAtk+lBpep4sK6tFf3/7eWN8fPSpQq4uCg7XHdXYIgmIiIiIiLq4wwNgUmTFJg0qTFYV1QA2dmaV6zz8jo+GvigQSo4OfXMAA0wRBMREREREVEzjI0Bb28FvL0bg3VZGZCV1RiqMzLq3zutjTFjFNDr+CDiXYYhmoiIiIiIiAQxMQF8fRXw9W0M1qWlwOXLjaE6I0MfRUUtp+Rx4xQtTusJGKKJiIiIiIio3UxNgSlTFJgypTEc378vwuXLjc9YZ2SIUVxcH6wZoomIiIiIiIh+Z/BgFQIC5AgIkKvLfv1VhMxMMUM0ERERERERUVssLFSYPl3e9ozdXA9+nJuIiIiIiIjo0WKIJiIiIiIiIhKIIZqIiIiIiIhIIIZoIiIiIiIiIoEYoomIiIiIiIgEYogmIiIiIiIiEoghmoiIiIiIiEgghmjqVsRicVc3gf6A+6R74f7ofrhPuh/uk+6H+6R74f7ofrhPehZRaWmpqqsbQURERERERNQT8Eo0ERERERERkUAM0UREREREREQCMUQTERERERERCcQQTURERERERCQQQzQRERERERGRQAzRRERERERERAIxRFOn+umnn/Dss8/C3t4eQ4cOxfTp0xEbG6tVHTU1Ndi2bRs8PT1hZWWFkSNHYvXq1bh7924ntbr3uXXrFnbt2oW5c+fC3d0dQ4YMwYgRI7BgwQJcvHhRcD3JyckwNTVt8ScqKqoTt6L38fDwaPFvGRQUpFVdX3zxBaZOnYqhQ4fCwcEBzz//PDIzMzun4b1UVFRUq/+/TU1NMXv27DbrYT/R3vHjx7FmzRr4+/vD0tKyzb9TeXk53njjDbi7u8PS0hIeHh546623UFlZqfW6JRIJAgMDYWtrCzs7OzzzzDNITEzsyOb0CkL3SV1dHeLi4rB8+XJMnDgRNjY2sLW1xbRp0/DZZ59BoVBotV5dHhd7G236yZYtW1o9DhUVFWm17vz8fCxevBjOzs6wtraGj48PPvvsM6hUfftNudrsk7bOL6ampiguLha0XvaTrqff1Q2g3ispKQnz5s3DgAEDEBISAiMjI5w6dQpLlixBcXExVq1a1WYdSqUSL774IiQSCSZMmIDZs2ejoKAAhw4dQmJiIr799ltYWFg8gq3p2fbu3YuPP/4YTk5OCAgIgIWFBQoKCnD69GmcPn0a//73vxESEiK4Ph8fH/zpT39qUu7h4aHLZvcJgwYNQkRERJNye3t7wXVs374dkZGRsLOzw5IlS1BZWYmvvvoKM2fORFxcHCZPnqzLJvdaHh4e2LBhQ7PTTp06hatXr2LatGmC62M/ES4yMhI3b96Eubk5rKyscPPmzRbnraqqQlBQELKzszF16lTMnz8fWVlZ2LFjB1JSUnDmzBkMGDBA0HqPHz+O8PBwWFhYIDQ0FAAQGxuL4OBgfP7555gzZ45Otq8nErpPCgsLsWjRIhgZGcHPzw9PP/00ysvLcfbsWaxbtw7nzp3DsWPHIBKJBK9bF8fF3kibftIgNDS02b+biYmJ4PXm5OTgqaeeQnV1NYKDg/H444/j3LlzWLduHXJycvD+++9rtR29iTb7pKXzS2FhIb744guMHDkStra2gtfNftK1RKWlpX37KyTqFHK5HBMmTMCtW7eQkJCA0aNHAwDKysowbdo03LhxAxcvXmyzox85cgQrV67E/PnzsW/fPvVJeP/+/Vi7di0WL16Mjz/+uLM3p8c7deoUBg8e3OQDfWpqKubMmQNDQ0Pk5uaif//+rdaTnJyMWbNmYcOGDXj99dc7s8l9QkOYys7ObncdBQUFmDRpEhwdHSGRSNQfjLKysjBjxgw4OjoiLS0Nenq88ai9amtrMXLkSJSXl0MqlcLS0rLV+dlPtHfhwgU4OzvD3t4eH330ETZv3oydO3ciLCysybz//Oc/8d5772HNmjXYtGmTunzTpk34+OOPsXHjRqxdu7bNdZaWlmLMmDHQ19dHUlISbGxsAAAlJSXw8/MDAGRmZsLY2Fg3G9nDCN0nt27dwpkzZxAaGgpDQ0N1eVVVFZ555hlkZGTg888/R3BwsKD16uK42Ftp00+2bNmCbdu2IT4+Hr6+vh1ab2BgIFJTUxETE4MZM2YAqD8uzpkzB2lpaTh37hwmTpzYoXX0VNrsk5asX78e+/btQ2RkJFauXCloGfaTrsdPVdQpkpKSUFhYiPnz56sDNFD/zefatWtRW1uLo0ePtlnPoUOHAAAbN27U+BZ7yZIlcHR0RExMDB4+fKj7DehlZs+e3ewVMW9vb/j6+qK0tBRSqbQLWkYdFRUVBblcjnXr1mlcWRg9ejTmzZuH3NxcpKWldWELe77Tp0/j/v37mDlzZpsBmtrH399f0NUTlUqFw4cPw8jICOvXr9eYtn79ehgZGanPG205efIkysrK8PLLL6sDNADY2Nhg2bJluHfvHr7++mvtNqQXEbpPhg4diqVLl2oEaAAwNDTEihUrAAApKSmd0sa+Rug+0aX8/HykpqbC19dXHaABoF+/fvj73/8OADh48OAjbVN30tF9Ul1djZiYGPTr1w8vvPCCDltGnY23c1On+O9//wsAmDp1apNpDbdDtnVSra6uxsWLFzF8+PAmByiRSISAgAAcOHAAGRkZ8Pb21lHL+x4DAwMAgFgsFrzMtWvXsGvXLlRXV2Po0KHw8/PD0KFDO6uJvVptbS2ioqJw+/ZtGBsbw9PTE08++aTg5dvqa9HR0UhJSYGPj4/O2tzXNISyhQsXarUc+4nuFRQU4Oeff8a0adOaDW2TJk2CRCJBcXFxm7dFttV3tm7dipSUFPVt3qS99pxfgI4fF6lRamoqLl26BD09PTg7O8Pf3x9GRkaCl2+tn3h5ecHQ0JBfknRAfHw8SktLMWfOHK0fT2Q/6VoM0dQpCgoKAAAuLi5NpllZWcHIyAjXrl1rtY7CwkIolUo4Ozs3O72hvKCggCG6nW7evIkLFy7A2toao0aNErxcTEwMYmJi1L/r6+vj5Zdfxrvvvqv1h6W+7s6dO+qrNQ08PT3x2WefwcnJqc3lCwoKYGRkBCsrqybTGvpfQ38k7d24cQOJiYmwsbHB9OnTtVqW/UT3Gv4vt3ZekEgkKCgoaDNEt3aeYt/RjSNHjgBoPoC1pqPHRWq0ZcsWjd9NTEywdetWwV8OtdbnxGIxHBwckJOTA7lcDn19xgptHT58GID2X9IC7CddjbdzU6coLy8HUD/oQXOMjY3V87RVR0uDXzTU3VY91Ly6ujqEh4ejpqYGmzZtEvSh3sLCAps2bUJaWhpKSkqQl5eHqKgoODs7Y9euXdi4ceMjaHnvERYWhri4OOTl5eHWrVtISkrC888/j59++gmzZ89GRUVFm3WUl5e32s8a5qH2iYqKglKpRGhoqODgy37SeXR5XmjtPMW+03Gff/45EhIS4Ofnh6eeekrwcro4LhLg7u6OTz75BJmZmbh9+zYuX76M9957DyKRCK+88grOnDkjqJ62+pyxsTGUSmW7Rsbv665fv47k5GTY2toiICBAq2XZT7oevzIi6oOUSiVeeeUVpKamYtGiRYKfw3F1dYWrq6v6d0NDQwQFBeHJJ5+Ej48P9uzZgzVr1mDIkCGd1fRe5W9/+5vG76NHj8aePXsA1I8afPDgQcGDjJDuKZVKREVFQSQS4c9//rPg5dhPqK87e/Ys1q9fDzs7O+zdu1erZXlc1I1Zs2Zp/O7g4ICXX34ZTzzxBIKDgxEZGYnAwMAuah0B9XdqqFQqhIWFaT34J/tJ1+OVaOoUbV0NqKioaPHq2R/rKCsra3Z6W1e7qXlKpRIrVqxATEwMnnvuOXz00UcdrtPKygqBgYGQy+VavXeamrdkyRIAQHp6epvzDho0qNV+1jAPae/ChQsoLi6Gn58fHB0dO1wf+0nH6fK80Np5in2n/c6dO4dFixbB0tIS8fHxsLa21km92hwXqWVTpkyBk5MTpFKpoDst2upzFRUVEIlEWj1nTfWfxY4ePQo9PT2tvqRtC/vJo8MQTZ2itefJ7ty5g8rKyhafaWvg6OgIPT29Fp+dbihv7nk2al7DFeijR49i/vz52L17t85efWRubg4AePDggU7q68u0+Vu6uLigsrISd+7caTKttWc+qW3tHVCsNewnHdPwf1kX54XWzlPsO+3zzTffYMGCBTA3N0d8fLxOvnxqwL6jOw1/SyFvN2mtzykUChQVFcHBwYHPQ2vp22+/RUlJCQICAmBnZ6ezetlPHh2GaOoUDSMBf/fdd02mSSQSjXlaMnDgQIwfPx55eXm4ceOGxjSVSoXz58/D0NAQ48aN01Gre7eGAH3s2DGEhIRgz549Oh3cqOHK2qN+/UZvpM3fUhd9jZq6f/8+zpw5AzMzMzzzzDM6q5f9pGNcXFzw+OOPIz09HVVVVRrTqqqqkJ6eDgcHhzYHFQPYd3Ttm2++wcKFC2FmZob4+Pg2vyjXFvuOblRVVSEnJweGhobqwNWa1vpJWloaqqqq2E/aoSMDirWG/eTRYYimTjFlyhQ4Ojriyy+/RFZWlrq8rKwMH374YZP34d2+fRsymazJ7UKLFi0CALzzzjtQqVTq8gMHDuD69et49tlnMXDgwE7emp6v4RbuY8eOITg4GHv37m01QN+7dw8ymQz37t3TKM/MzGx2/t27dyM5ORkuLi7w9PTUZdN7LZlM1uw3xTKZDJs2bQIAzJ8/X11eVlYGmUyG27dva8wfFhYGfX19fPDBBxr9JysrCydOnMATTzwBLy+vztmIXuzYsWOora3Fc889h/79+zc7D/vJoycSibBgwQJUVlbi/fff15j2/vvvo7KyUn3eaPDgwQPIZDLcvHlTo3zu3LkYNGgQ9u7di5KSEnV5SUkJ9u3bB3Nzc51+gdKbJSQkYOHChTA1NUV8fHybV/Dr6uogk8lQWFioUa7tcZGaV1FRgfz8/CblDx8+xOrVq1FRUYHg4OAmV49lMhlkMplG2fDhw+Ht7Y3k5GQkJCSoy2tra/GPf/wDgO6DYG/366+/4uzZs7CwsMDTTz/d4nzsJ92bqLS0VNX2bETaS0pKwrx58zBgwACEhITAyMgIp06dws2bN/Huu+9i1apV6nkjIiJw9OhR7Ny5E2FhYepypVKJZ599FhKJBBMmTICPjw+uXbuG+Ph42NvbQyKRaP1evb5oy5Yt2LZtG4yMjLB8+fJmA3RQUBBGjx6tMf+GDRvw+uuvq+fx8PCAgYEBxo0bh6FDh+LBgwf48ccfkZWVBRMTE3z11VcYP378I9uunmzLli3YtWsXvL29YWdnh8ceewz5+flISEhAXV0d1q5dqzGKc1RUFFasWIHQ0FDs3r1bo67t27cjMjISdnZ2mD17NiorK/HVV1+htrYWcXFxmDx58qPevB7P29sbUqkUKSkpLb7+jf1Edw4dOoS0tDQAgFQqxeXLlzF58mT1a1q8vLzUH9Srqqowc+ZMXLlyBVOnTsWYMWNw+fJlfPfdd/D09MTp06c1vlxNTk7GrFmz4OPjg9OnT2us9/jx4wgPD4eFhQXmzp0LAIiNjcW9e/dw4MABBAcHP4Kt756E7hOZTAZfX1/U1NRg3rx5GDZsWJO67O3tNc7tRUVFGDNmDOzs7JCdna0u1/a42NcI3SdFRUUYO3YsPD09MWLECFhZWeGXX35BYmIiSkpK4Obmhq+//hqDBw/WqN/U1BQAUFpaqlF+9epVzJw5E9XV1Zg7dy6sra1x7tw5XL16FcuWLWvyhVZfos2xq8GOHTvw1ltvYcWKFeovIprDftK98QEG6jR+fn44e/YstmzZgtjYWNTV1cHNzQ2bN29GSEiIoDr09PQQHR2Njz76CMePH8euXbtgZmaGBQsW4M0332SAFqjhdvjKykps37692Xns7e3VIbolL730EiQSCVJTU3H//n3o6enBzs4OERERWLlyJWxsbHTe9t7K19cXMpkMWVlZSEtLw4MHD2Bubo4ZM2Zg6dKlWr1X9dVXX4W9vT12796N/fv3w8DAAF5eXnjjjTcwduzYztuIXurSpUuQSqUYP368Vu9Pb8B+or20tDQcPXpUo+z777/H999/r/694YOooaEhTp8+ja1btyI+Ph7JycmwsrLCypUrsWHDBq3uTnr++edhbm6ODz74ANHR0RCJRBgzZgzWr18Pf39/nWxbTyV0n9y5cwc1NTUAgBMnTjRbl4+Pj0aIbokuj4u9kdB9YmZmhqVLl+LSpUtISEhAaWkpBg4ciBEjRiA8PBzLli3Tqp+4urpCIpEgMjIS586dw4MHD+Di4oLt27fjpZde0tn29UTaHLsaNLw/vb1X8NlPugdeiSYiIiIiIiISiM9EExEREREREQnEEE1EREREREQkEEM0ERERERERkUAM0UREREREREQCMUQTERERERERCcQQTURERERERCQQQzQRERERERGRQAzRRERERERERAIxRBMREREREREJxBBNREREPVZRURFMTU1hamra1U0hIqI+Qr+rG0BERESdKygoCCkpKYLmLS0t7dzGEBER9XAM0URERH2Era0tbG1tu7oZREREPRpDNBERUR8RFhaG119/vaubQURE1KPxmWgiIiIiIiIigRiiiYiIqIk/Dtj1n//8B0FBQXBwcICNjQ2mT5+OL774otU6cnNzsWLFCnh4eMDS0hIODg4IDAzEoUOHoFAoWlxOLpcjOjoaISEhGDZsGCwtLeHq6orAwEDs2LEDZWVlLS6blpaG5557Dk5OTrC2toa3tzf27t0LlUrV7PyZmZlYtmwZ3N3dYWlpCRsbG3h4eGDevHnYsWNHi8sREVHfxdu5iYiIqFV79uzBhg0bYGZmBmdnZ5SUlODixYvqn/fee6/JMrGxsQgPD0dtbS0MDQ3h5uaG3377DampqUhNTUVsbCyio6MxcOBAjeXu3buHF198Eenp6QAAKysruLu74+7du0hPT0dqairGjh0LX1/fJuuMiorCqlWrYGJiAkdHR9y8eRNSqRSvvfYabty4gcjISI35v/32W4SGhqKurg5GRkYYNmwY9PX1cevWLUgkEkgkEkREREBfnx+XiIioEa9EExERUavefPNNrF+/Hnl5eTh//jxyc3Px4YcfQk9PD3v37sXJkyc15pfJZIiIiEBtbS0WLlwImUyGCxcu4PLlyzh58iQGDRqE8+fPY+PGjRrLqVQqLFq0COnp6bC1tUVcXBxyc3Px3XffITs7G9euXcMHH3wAS0vLZtu5du1aREZGIj8/H+fPn0d+fr56HTt37kRhYaHG/Js2bUJdXR1Wr16NvLw8pKamIikpCfn5+cjOzsbmzZuhp8ePSkREpElUWlrK+5SIiIh6MaGvuAoMDER0dDSA+tu5x4wZAwAICAhAbGxsk/lXr16NgwcPws3NDampqerylStX4siRI3Bzc0NKSgpEIpHGcocOHcJf/vIXGBgYIDs7G9bW1gDqbxkPDQ1F//79kZSUhCeeeKLNNv++nQsWLMCOHTuazOPt7Q2pVIqtW7di+fLl6nIrKyvU1NSgqKgIJiYmba6LiIgI4JVoIiKiPsPW1haTJ09u8WfkyJHNLhcREdFquVQqRXFxsbo8ISEBALB8+fImARoAQkNDMWTIENTV1eH8+fPq8lOnTgEAZs2aJShA/9HSpUubLZ84cSIA4Nq1axrldnZ2AIAvv/xS63UREVHfxYd8iIiI+oj2vuLK1dW12fLhw4dDX18fcrkcMpkMtra2KCsrw507dwAAbm5uzS5nYGCA4cOH4+7du8jLy1OXS6VSAI2hV1vDhg1rtnzIkCEAgMrKSo3y1atXY9WqVVi3bh0++eQTBAQEYMKECfDx8YG9vX272kBERL0fr0QTERFRq1p6BlksFmPw4MEAgIqKCgCaQbWl5QCob+FuWO73/27vrdWGhobNljc81/zHkbYXLFiAw4cPY9KkSSgqKsL+/fsRERGB0aNHY/r06UhOTm5XO4iIqHdjiCYiIqJW/fLLL82WKxQK3L9/HwBgbGwMADAyMmpzOQC4ffu2xnK//3drr7DStVmzZuGbb75BYWEhYmJi8Ne//hVOTk64ePEi5s2bh+zs7EfWFiIi6hkYoomIiKhVOTk5zZbn5eVBLpcDAEaMGAGg/iqylZUVgMbbs/9ILperb+NuWA4ARo0aBQD44YcfdNNwLZiYmGDGjBl4++238eOPP2LChAmora3FoUOHHnlbiIioe2OIJiIiolZ9+umnrZa7ubnB1tZWXf7UU0+pp//xFmoAOHbsGO7evQsDAwMEBASoy+fMmQMAiI+P13hW+lHT19fH+PHjAQA///xzl7WDiIi6J4ZoIiIialVSUhK2bdumvuqsUqlw8OBBHD58GADw6quvasy/atUqDBgwAFKpFGvWrEFVVZV6WmJiIt58800AwOLFi9VXrQFg5syZ8PPzQ01NDUJCQpo8k1xeXo79+/cjNze3w9tUXl6ORYsWQSKRoLa2VmNaZmam+pVenp6eHV4XERH1Lhydm4iIqI+IiopCYmJiq/Ns27ZN/d7lBpGRkdiwYQM+/fRTODk5oaSkRD0C99KlSxESEqIx/4gRI7B7926Eh4fj4MGDOHHiBIYPH47ffvsN169fB1D/7ul33nmnyfoPHDiAF154AT/++CNmzZoFa2tr2NjY4O7duygpKYFCoUB8fHy7XoH1e0qlEnFxcYiLi0O/fv3g7OwMQ0ND3L17Fzdu3AAAPPnkkxrvlSYiIgIYoomIiPqM4uJijfc5N6e8vLxJWXh4OOzs7LBz505kZ2dDLpdj/PjxWLZsGV544YVm65k7dy5cXV3xr3/9C0lJSfjf//6HAQMGwMvLC6GhoQgLC4NYLG6ynLm5Oc6cOYOjR48iJiYGV65cQXZ2NiwsLDB58mQEBQU1CfntYWxsjH379iEpKQmXLl3C7du3UVZWBmNjY3h5eWHu3LlYvHgx+vXr1+F1ERFR7yIqLS1t+rASERER9WlFRUXqsFpaWtq1jSEiIupG+Ew0ERERERERkUAM0UREREREREQCMUQTERERERERCcQQTURERERERCQQBxYjIiIiIiIiEohXoomIiIiIiIgEYogmIiIiIiIiEoghmoiIiIiIiEgghmgiIiIiIiIigRiiiYiIiIiIiARiiCYiIiIiIiISiCGaiIiIiIiISCCGaCIiIiIiIiKBGKKJiIiIiIiIBPo/XlncUOINHzsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1000x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = sbs_doc_emb.plot_losses()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1ZpLkF80PIpj",
    "outputId": "9d3514ef-83e7-44ac-cf94-3483edca7f20"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[442, 453],\n",
       "        [295, 318]])"
      ]
     },
     "execution_count": 137,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "StepByStep.loader_apply(test_loader, sbs_doc_emb.correct)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kLKZdrC9PIpj"
   },
   "source": [
    "# BERT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PtgTF0CIPIpk",
    "outputId": "55014197-ca11-4f2c-bbb5-e4a28cc468df"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'transformers.models.bert.modeling_bert.BertModel'>\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoModel\n",
    "auto_model = AutoModel.from_pretrained('bert-base-uncased')\n",
    "print(auto_model.__class__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {
    "id": "bYjuuJG-PIpk"
   },
   "outputs": [],
   "source": [
    "from transformers import BertModel\n",
    "bert_model = BertModel.from_pretrained('bert-base-uncased')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IpDTWGNBPIpk",
    "outputId": "f13d56d3-1027-4234-9c31-333b981121cc"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BertConfig {\n",
       "  \"_name_or_path\": \"bert-base-uncased\",\n",
       "  \"architectures\": [\n",
       "    \"BertForMaskedLM\"\n",
       "  ],\n",
       "  \"attention_probs_dropout_prob\": 0.1,\n",
       "  \"classifier_dropout\": null,\n",
       "  \"gradient_checkpointing\": false,\n",
       "  \"hidden_act\": \"gelu\",\n",
       "  \"hidden_dropout_prob\": 0.1,\n",
       "  \"hidden_size\": 768,\n",
       "  \"initializer_range\": 0.02,\n",
       "  \"intermediate_size\": 3072,\n",
       "  \"layer_norm_eps\": 1e-12,\n",
       "  \"max_position_embeddings\": 512,\n",
       "  \"model_type\": \"bert\",\n",
       "  \"num_attention_heads\": 12,\n",
       "  \"num_hidden_layers\": 12,\n",
       "  \"pad_token_id\": 0,\n",
       "  \"position_embedding_type\": \"absolute\",\n",
       "  \"transformers_version\": \"4.38.1\",\n",
       "  \"type_vocab_size\": 2,\n",
       "  \"use_cache\": true,\n",
       "  \"vocab_size\": 30522\n",
       "}"
      ]
     },
     "execution_count": 140,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bert_model.config"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fBkvBjp5PIpk"
   },
   "source": [
    "## Tokenization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "M7ZcsRmdPIpk",
    "outputId": "2b763184-b53a-4abe-8f9d-c152173b8145",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "30522"
      ]
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from transformers import BertTokenizer\n",
    "bert_tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')\n",
    "len(bert_tokenizer.vocab)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "kyF47gBWPIpk",
    "outputId": "b75dac53-6d84-4717-ced7-e32aa06122c9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[  101,  5650,  2003,  1999, 10288, 24759,  5555,  6321,  2206,  1996,\n",
       "          2317, 10442,   102,  3582,  1996,  2317, 10442,  1010,  9253,   102]]), 'token_type_ids': tensor([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]])}"
      ]
     },
     "execution_count": 142,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence1 = 'Alice is inexplicably following the white rabbit'\n",
    "sentence2 = 'Follow the white rabbit, Neo'\n",
    "tokens = bert_tokenizer(sentence1, sentence2, return_tensors='pt')\n",
    "tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "brcF_Ox1PIpl",
    "outputId": "f76b3dbf-99c5-4ee8-e12b-bc057e1abe0a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['[CLS]', 'alice', 'is', 'in', '##ex', '##pl', '##ica', '##bly', 'following', 'the', 'white', 'rabbit', '[SEP]', 'follow', 'the', 'white', 'rabbit', ',', 'neo', '[SEP]']\n"
     ]
    }
   ],
   "source": [
    "print(bert_tokenizer.convert_ids_to_tokens(tokens['input_ids'][0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1Cjrk1qCPIpl",
    "outputId": "bbe1e67a-81e0-4dcf-d026-f68ae4011738"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'transformers.models.bert.tokenization_bert_fast.BertTokenizerFast'>\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoTokenizer\n",
    "auto_tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')\n",
    "print(auto_tokenizer.__class__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j0A5jpZJPIpl"
   },
   "source": [
    "## Input Embeddings\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/bert_input_embed.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "JOhc49UmPIpl",
    "outputId": "dac827b7-6fbd-4730-fdae-5400dc2c884c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BertEmbeddings(\n",
       "  (word_embeddings): Embedding(30522, 768, padding_idx=0)\n",
       "  (position_embeddings): Embedding(512, 768)\n",
       "  (token_type_embeddings): Embedding(2, 768)\n",
       "  (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "  (dropout): Dropout(p=0.1, inplace=False)\n",
       ")"
      ]
     },
     "execution_count": 145,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_embeddings = bert_model.embeddings\n",
    "input_embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Zo5g1KN2PIpm",
    "outputId": "bd8ff74b-0b9a-4639-bc78-65435142e1a6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Embedding(30522, 768, padding_idx=0)"
      ]
     },
     "execution_count": 146,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "token_embeddings = input_embeddings.word_embeddings\n",
    "token_embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "jr_sA4KEPIpm",
    "outputId": "5f0e83c4-7c3d-44d9-aa10-4fd886ae3f37"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 1.3630e-02, -2.6490e-02, -2.3503e-02,  ...,  8.6805e-03,\n",
       "           7.1340e-03,  1.5147e-02],\n",
       "         [-6.9710e-02, -8.8202e-02,  5.0619e-03,  ...,  1.4105e-02,\n",
       "           2.1815e-02, -1.3769e-02],\n",
       "         [-3.6044e-02, -2.4606e-02, -2.5735e-02,  ...,  3.3691e-03,\n",
       "          -1.8300e-03,  2.6855e-02],\n",
       "         ...,\n",
       "         [ 5.2089e-05, -1.0468e-02, -9.9103e-03,  ...,  1.4558e-02,\n",
       "           1.3217e-02,  2.2406e-02],\n",
       "         [-3.5037e-02, -7.2933e-02, -3.6124e-02,  ..., -5.7723e-02,\n",
       "          -5.5074e-03,  7.2688e-03],\n",
       "         [-1.4521e-02, -9.9615e-03,  6.0263e-03,  ..., -2.5035e-02,\n",
       "           4.6379e-03, -1.5378e-03]]], grad_fn=<EmbeddingBackward0>)"
      ]
     },
     "execution_count": 147,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_token_emb = token_embeddings(tokens['input_ids'])\n",
    "input_token_emb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rx5LiAACPIpm",
    "outputId": "fe1b4a26-8173-4779-b48a-ec4e311609f0"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Embedding(512, 768)"
      ]
     },
     "execution_count": 148,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "position_embeddings = input_embeddings.position_embeddings\n",
    "position_embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "W6e0kmEnPIpm",
    "outputId": "44ac81b9-0e27-4e1c-9761-728bc8c3b4ee"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,\n",
       "          14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,\n",
       "          28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,\n",
       "          42,  43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,\n",
       "          56,  57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,\n",
       "          70,  71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,\n",
       "          84,  85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,\n",
       "          98,  99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,\n",
       "         112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125,\n",
       "         126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139,\n",
       "         140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153,\n",
       "         154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167,\n",
       "         168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181,\n",
       "         182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195,\n",
       "         196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209,\n",
       "         210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223,\n",
       "         224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237,\n",
       "         238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251,\n",
       "         252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265,\n",
       "         266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279,\n",
       "         280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293,\n",
       "         294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307,\n",
       "         308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321,\n",
       "         322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335,\n",
       "         336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349,\n",
       "         350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363,\n",
       "         364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377,\n",
       "         378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391,\n",
       "         392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405,\n",
       "         406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419,\n",
       "         420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433,\n",
       "         434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447,\n",
       "         448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461,\n",
       "         462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475,\n",
       "         476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489,\n",
       "         490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503,\n",
       "         504, 505, 506, 507, 508, 509, 510, 511]])"
      ]
     },
     "execution_count": 149,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "position_ids = torch.arange(512).expand((1, -1))\n",
    "position_ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "SqZUNJBhPIpn",
    "outputId": "4621291c-aa23-4dbd-9b4b-5deaeef3201b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 1.7505e-02, -2.5631e-02, -3.6642e-02,  ...,  3.3437e-05,\n",
       "           6.8312e-04,  1.5441e-02],\n",
       "         [ 7.7580e-03,  2.2613e-03, -1.9444e-02,  ...,  2.8910e-02,\n",
       "           2.9753e-02, -5.3247e-03],\n",
       "         [-1.1287e-02, -1.9644e-03, -1.1573e-02,  ...,  1.4908e-02,\n",
       "           1.8741e-02, -7.3140e-03],\n",
       "         ...,\n",
       "         [-9.2809e-03,  8.3268e-03, -4.1643e-03,  ...,  3.4903e-02,\n",
       "          -1.8319e-02, -2.9017e-03],\n",
       "         [-8.5999e-03,  3.2205e-04, -2.1249e-03,  ...,  2.7744e-02,\n",
       "          -7.2760e-03, -2.0280e-03],\n",
       "         [-3.4622e-04, -8.3709e-04, -2.2228e-02,  ...,  2.3493e-02,\n",
       "          -4.5198e-04, -5.7741e-04]]], grad_fn=<EmbeddingBackward0>)"
      ]
     },
     "execution_count": 150,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seq_length = tokens['input_ids'].size(1)\n",
    "input_pos_emb = position_embeddings(position_ids[:, :seq_length])\n",
    "input_pos_emb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "5DXX4yiEPIpn",
    "outputId": "2a3ff3bd-2faf-4b91-840b-f1c29b9c98bd"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Embedding(2, 768)"
      ]
     },
     "execution_count": 151,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "segment_embeddings = input_embeddings.token_type_embeddings\n",
    "segment_embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "GgCRoV2sPIpn",
    "outputId": "a764c7b3-3e5d-4475-f4ed-b9cf33a49e46"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 0.0004,  0.0110,  0.0037,  ..., -0.0066, -0.0034, -0.0086],\n",
       "         [ 0.0004,  0.0110,  0.0037,  ..., -0.0066, -0.0034, -0.0086],\n",
       "         [ 0.0004,  0.0110,  0.0037,  ..., -0.0066, -0.0034, -0.0086],\n",
       "         ...,\n",
       "         [ 0.0011, -0.0030, -0.0032,  ...,  0.0047, -0.0052, -0.0112],\n",
       "         [ 0.0011, -0.0030, -0.0032,  ...,  0.0047, -0.0052, -0.0112],\n",
       "         [ 0.0011, -0.0030, -0.0032,  ...,  0.0047, -0.0052, -0.0112]]],\n",
       "       grad_fn=<EmbeddingBackward0>)"
      ]
     },
     "execution_count": 152,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_seg_emb = segment_embeddings(tokens['token_type_ids'])\n",
    "input_seg_emb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "iUkjZrKTPIpn",
    "outputId": "da73d7a9-e2a3-4fc8-97e5-97adc088fbfb"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 0.0316, -0.0411, -0.0564,  ...,  0.0021,  0.0044,  0.0219],\n",
       "         [-0.0615, -0.0750, -0.0107,  ...,  0.0364,  0.0482, -0.0277],\n",
       "         [-0.0469, -0.0156, -0.0336,  ...,  0.0117,  0.0135,  0.0109],\n",
       "         ...,\n",
       "         [-0.0081, -0.0051, -0.0172,  ...,  0.0542, -0.0103,  0.0083],\n",
       "         [-0.0425, -0.0756, -0.0414,  ..., -0.0252, -0.0180, -0.0060],\n",
       "         [-0.0138, -0.0138, -0.0194,  ...,  0.0032, -0.0011, -0.0133]]],\n",
       "       grad_fn=<AddBackward0>)"
      ]
     },
     "execution_count": 153,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_emb = input_token_emb + input_pos_emb + input_seg_emb\n",
    "input_emb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dGkuZ7UdPIpn"
   },
   "source": [
    "## Pretraining Tasks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m2FoG2AGPIpn"
   },
   "source": [
    "### Masked Language Model (MLM)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/bert_mlm.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ZTw0eDKePIpo",
    "outputId": "4a5e6ca2-3c65-4798-ff2a-b3423cc6240a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[101, 5650, 2003, 1999, 10288, 24759, 5555, 6321, 2206, 1996, 2317, 10442, 102]"
      ]
     },
     "execution_count": 154,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence = 'Alice is inexplicably following the white rabbit'\n",
    "tokens = bert_tokenizer(sentence)\n",
    "tokens['input_ids']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3yCbwZXgPIpo",
    "outputId": "387b400d-3428-4ad8-b313-d5770a10d089"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[  101,  5650,  2003,  1999, 10288, 24759,   103,  6321,  2206,  1996,\n",
       "          2317, 10442,   102]]), 'token_type_ids': tensor([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]), 'labels': tensor([[-100, -100, -100, -100, -100, -100, 5555, -100, -100, -100, -100, -100,\n",
       "         -100]])}"
      ]
     },
     "execution_count": 155,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from transformers import DataCollatorForLanguageModeling\n",
    "torch.manual_seed(41)\n",
    "data_collator = DataCollatorForLanguageModeling(tokenizer=bert_tokenizer, mlm_probability=0.15)\n",
    "mlm_tokens = data_collator([tokens])\n",
    "mlm_tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "XI180xPwPIpo",
    "outputId": "10bc403d-4019-4606-8c9b-332f3572921a",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['[CLS]', 'alice', 'is', 'in', '##ex', '##pl', '##ica', '##bly', 'following', 'the', 'white', 'rabbit', '[SEP]']\n",
      "['[CLS]', 'alice', 'is', 'in', '##ex', '##pl', '[MASK]', '##bly', 'following', 'the', 'white', 'rabbit', '[SEP]']\n"
     ]
    }
   ],
   "source": [
    "print(bert_tokenizer.convert_ids_to_tokens(tokens['input_ids']))\n",
    "print(bert_tokenizer.convert_ids_to_tokens(mlm_tokens['input_ids'][0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gyTICtkbPIpo"
   },
   "source": [
    "### Next Sentence Prediction (NSP)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/bert_nsp.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "eWrQN9DpPIpo",
    "outputId": "01423718-f54a-4989-850c-f570f399c2f5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BertPooler(\n",
       "  (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "  (activation): Tanh()\n",
       ")"
      ]
     },
     "execution_count": 157,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bert_model.pooler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "WmVOtEwePIpp",
    "outputId": "21e84db6-9fb7-4316-de40-1a3e9beec726"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[  101,  5650,  4076,  1996,  2317, 10442,   102,  3582,  1996,  2317,\n",
       "         10442,  9253,   102]]), 'token_type_ids': tensor([[0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]])}"
      ]
     },
     "execution_count": 158,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence1 = 'alice follows the white rabbit'\n",
    "sentence2 = 'follow the white rabbit neo'\n",
    "bert_tokenizer(sentence1, sentence2, return_tensors='pt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_HkUpyLUPIpp"
   },
   "source": [
    "## Outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "5C-AaHkFPIpp",
    "outputId": "efcf2e58-a795-49c2-d507-acdae8f94095"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      },
      "text/plain": [
       "'So the Woodman went at once to the trees and began to work; and he soon made a truck out of the limbs of trees, from which he chopped away all the leaves and branches.'"
      ]
     },
     "execution_count": 159,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence = train_dataset[0]['sentence']\n",
    "sentence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "AmqPWjJuPIpp",
    "outputId": "341934d6-5091-47d9-f4fb-2a2bffb6ff94"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[  101,  2061,  1996,  3536,  2386,  2253,  2012,  2320,  2000,  1996,\n",
       "          3628,  1998,  2211,  2000,  2147,  1025,  1998,  2002,  2574,  2081,\n",
       "          1037,  4744,  2041,  1997,  1996, 10726,  1997,  3628,  1010,   102]]), 'token_type_ids': tensor([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "         1, 1, 1, 1, 1, 1]])}"
      ]
     },
     "execution_count": 160,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokens = bert_tokenizer(sentence,\n",
    "                        padding='max_length',\n",
    "                        max_length=30,\n",
    "                        truncation=True,\n",
    "                        return_tensors=\"pt\")\n",
    "tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "f_ssmrA7PIpp",
    "outputId": "5835486d-41be-4729-ca38-9fe74dde024d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "odict_keys(['last_hidden_state', 'pooler_output', 'hidden_states', 'attentions'])"
      ]
     },
     "execution_count": 161,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bert_model.eval()\n",
    "out = bert_model(input_ids=tokens['input_ids'],\n",
    "                 attention_mask=tokens['attention_mask'],\n",
    "                 output_attentions=True,\n",
    "                 output_hidden_states=True,\n",
    "                 return_dict=True)\n",
    "out.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "n0lNYShtPIpq"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/bert_embeddings.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "O9qGa0qaPIpq",
    "outputId": "f2067bfe-3715-4247-9cbb-b63199d76a28"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.5442, -0.0137, -0.8150,  ...,  0.1782,  0.8116,  0.2801],\n",
       "        [ 0.6964,  0.2052, -0.1539,  ...,  0.3547,  0.7251, -0.6998],\n",
       "        [ 1.3538,  0.1839, -0.2409,  ..., -0.0459,  0.0591, -0.9654],\n",
       "        ...,\n",
       "        [ 0.0936,  0.8914,  0.7814,  ..., -0.4037,  0.1226, -0.1378],\n",
       "        [ 0.1062,  1.1001,  0.7522,  ..., -0.2057, -0.1345, -0.6589],\n",
       "        [-0.3197, -0.4288, -0.3550,  ...,  0.2808,  0.1968, -0.3568]],\n",
       "       grad_fn=<SliceBackward0>)"
      ]
     },
     "execution_count": 162,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "last_hidden_batch = out['last_hidden_state']\n",
    "last_hidden_sentence = last_hidden_batch[0]\n",
    "# Removes hidden states for [PAD] tokens using the mask\n",
    "mask = tokens['attention_mask'].squeeze().bool()\n",
    "embeddings = last_hidden_sentence[mask]\n",
    "# Removes embeddings for the first [CLS] and last [SEP] tokens\n",
    "embeddings[1:-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rlqC7lDSPIpq",
    "outputId": "963e5271-968d-4acf-cc9e-44a2d1f689a1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.5181, -0.1753, -0.9000,  ...,  0.1124,  0.7104,  0.2835],\n",
       "        [ 0.6012,  0.1941, -0.2260,  ...,  0.3882,  0.6763, -0.5992],\n",
       "        [ 1.3117, -0.1014, -0.3750,  ..., -0.0536,  0.1620, -0.9091],\n",
       "        ...,\n",
       "        [-0.0166,  1.1062,  0.2548,  ..., -0.0166,  0.4479,  0.6551],\n",
       "        [ 0.0917,  0.6371,  0.5431,  ...,  0.0217,  0.0575, -0.0405],\n",
       "        [ 0.8255,  0.3259, -0.4444,  ...,  0.0391, -0.4220, -0.2392]],\n",
       "       device='cuda:0')"
      ]
     },
     "execution_count": 163,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_embeddings(bert_flair, sentence)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hNKAkRLyPIpq",
    "outputId": "f8897916-af98-45b9-dff1-a9770a8bfbfd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "13\n",
      "torch.Size([1, 30, 768])\n"
     ]
    }
   ],
   "source": [
    "print(len(out['hidden_states']))\n",
    "print(out['hidden_states'][0].shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "451QgYohPIpq",
    "outputId": "cf58f55e-9516-403e-fd97-1931a0d5e7b0"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(True)"
      ]
     },
     "execution_count": 165,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(out['hidden_states'][0] == bert_model.embeddings(tokens['input_ids'])).all()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "93o1_w-EPIpr",
    "outputId": "11ad9dc8-1a9a-41bf-e8a6-ce4b3bcc2b4c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(True)"
      ]
     },
     "execution_count": 166,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(out['hidden_states'][-1] == out['last_hidden_state']).all()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bYA0p0XqPIpr",
    "outputId": "5834f438-6420-4432-bd00-0f14f9c6f727"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(True)"
      ]
     },
     "execution_count": 167,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(out['pooler_output'] == bert_model.pooler(out['last_hidden_state'])).all()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "jsdP4CnSPIpr",
    "outputId": "bb3f69f1-9675-413c-8224-ab480eec2555"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12\n",
      "torch.Size([1, 12, 30, 30])\n"
     ]
    }
   ],
   "source": [
    "print(len(out['attentions']))\n",
    "print(out['attentions'][0].shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PlX9GImRPIpr"
   },
   "source": [
    "## Model IV - Classifying using BERT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {
    "id": "zWB0IsJEPIpr"
   },
   "outputs": [],
   "source": [
    "class BERTClassifier(nn.Module):\n",
    "    def __init__(self, bert_model, ff_units, n_outputs, dropout=0.3):\n",
    "        super().__init__()\n",
    "        self.d_model = bert_model.config.dim\n",
    "        self.n_outputs = n_outputs\n",
    "        self.encoder = bert_model\n",
    "        self.mlp = nn.Sequential(\n",
    "            nn.Linear(self.d_model, ff_units),\n",
    "            nn.ReLU(),\n",
    "            nn.Dropout(dropout),\n",
    "            nn.Linear(ff_units, n_outputs)\n",
    "        )\n",
    "\n",
    "    def encode(self, source, source_mask=None):\n",
    "        states = self.encoder(input_ids=source,\n",
    "                              attention_mask=source_mask)[0]\n",
    "        cls_state = states[:, 0]\n",
    "        return cls_state\n",
    "\n",
    "    def forward(self, X):\n",
    "        source_mask = (X > 0)\n",
    "        # Featurizer\n",
    "        cls_state = self.encode(X, source_mask)\n",
    "        # Classifier\n",
    "        out = self.mlp(cls_state)\n",
    "        return out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rN86j0y9PIps"
   },
   "source": [
    "### Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {
    "id": "LLFm2rAtPIps"
   },
   "outputs": [],
   "source": [
    "def tokenize_dataset(hf_dataset, sentence_field, label_field, tokenizer, **kwargs):\n",
    "    sentences = hf_dataset[sentence_field]\n",
    "    token_ids = tokenizer(sentences, return_tensors='pt', **kwargs)['input_ids']\n",
    "    labels = torch.as_tensor(hf_dataset[label_field])\n",
    "    dataset = TensorDataset(token_ids, labels)\n",
    "    return dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 145,
     "referenced_widgets": [
      "70e214355ffd434dacdd6b959e04b4c7",
      "8b1ed30ccf3143228a3aea84f390d41e",
      "fc64554be0cc4dbcb7d17ef58fe55262",
      "ba38d898cea04cc29db7a49d3d3367d4",
      "abcd64d0a2ee4adeb749cc2fc34e3a6a",
      "4390ce9b054e4e44948a4afa1adfd214",
      "f8e511c8d57a46a39ea1a67deabd6bfb",
      "2ee47a205c904782bdef418c25b89838",
      "8ff6cd267a28487fb8cac4cb8724fc0e",
      "d1b6a2821d0d42069d6cf483eb634d0b",
      "10570d170bec405c8c15d80523a0717c",
      "c5c9de8c50f94005905df2e4d0242dc5",
      "5a6a8a704b744c479bcabb9d5a16c8e8",
      "03b0d4b80ee8400e8bb1ff96fe794139",
      "3ac42e6d69fc4ea581d0fd3d1f77ea47",
      "995ca1139e7f46fbac1c296c3dcda07a",
      "dddb342989e44ec0b0c9e40bb0ccf843",
      "efb5452b364d4c358faea6a451f55bce",
      "5a124cf7785e4eccafa4d1c7d3cd2942",
      "1809cd29e041406687b13786774d4880",
      "64fe242aa6184a7cbff04d4043267e05",
      "a37527a86b78442f98e2591652d9075f",
      "527d9fab58da45b1a9930c302c10d000",
      "de96be39783b47c19fe9e3050ad3382d",
      "484eaeabda91461093d8aac55b07b490",
      "4bb8c462785b4ee3a905ef77c73a7a90",
      "6b11860e26f94a3a8a71315334e4e3c6",
      "8a23f02ece4341f7900c82ab906874eb",
      "91d21b681ffb4ceaabd85f2867e60cbd",
      "c541c75959db47b89b78ef1e8ac06295",
      "0d25366a3d654d3fba85508bf309e568",
      "eb43725f0e22483989c5334facd46da0",
      "2dfc9768d7a8421da53d9c812e7d9e8a",
      "dd8a687326dc49feafc45355eb129d95",
      "19a2336a406a45bf9e990fa8ad9a49d4",
      "13a2a27f04f7451aa8be99bb139efb27",
      "2d3e56d6572c45c7895b14699de36d87",
      "0d27001bb0cb4b689a75fbea7f1efa4c",
      "44050ee5b91145a8b22e2bc7de3ba85a",
      "2db54785b3554d46811fde033a15dc17",
      "f28cbac2858e4e079790d7f24ff1d73d",
      "975c4167faed4d339bbc3f2eeae9f748",
      "2a76cc1a0b46464bbd65fc92c4a64cc5",
      "bf06dc9d10d247ec9971327d4dcc2b1a"
     ]
    },
    "id": "hvCj__6mPIps",
    "outputId": "bda37a30-57ac-439a-8e34-3d48894598e0"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "70e214355ffd434dacdd6b959e04b4c7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer_config.json:   0%|          | 0.00/28.0 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c5c9de8c50f94005905df2e4d0242dc5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "config.json:   0%|          | 0.00/483 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "527d9fab58da45b1a9930c302c10d000",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "dd8a687326dc49feafc45355eb129d95",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer.json:   0%|          | 0.00/466k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "auto_tokenizer = AutoTokenizer.from_pretrained('distilbert-base-uncased')\n",
    "tokenizer_kwargs = dict(truncation=True, padding=True, max_length=30, add_special_tokens=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81,
     "referenced_widgets": [
      "65fa5ad06c2841f4bf33c3d917dfe686",
      "e9c3ea0435a14ce29d1c86b313ba9a54",
      "d0465475c82e4484842703a19508ef59",
      "cb4b2c83ca8a43e18823bb1c3a79fb0a",
      "1f95a9be7f54444b88744c67e8da5a80",
      "ee79ca16bfe24e658376c11ddfda1124",
      "70ae6722b0d34f91b70f435b8162e878",
      "ee8c3960cb684a54adc10a0516f3fc75",
      "8a916a0fe6874b7a97453b0f0dc314cf",
      "e40b00b93fe14e5ba62613d5fa74acf8",
      "063a969f176941fa84a21569b4c6ba2b",
      "67c086e6a2e347ce8378a7ea4041b27d",
      "65b52af198f04e0f8500eed14d3d4f8e",
      "635101e9dd8a4448bc74a0ed4316c507",
      "43bdb3e2257745268c506b8051afb03c",
      "e175619272a145949a2ec548819dd4eb",
      "43e354bd826a4538a1bc08972f52f711",
      "aeea02bd03be406bb2b047e3e9b82f95",
      "a6e572e7ab67495b8a5b4b1a50af8e0b",
      "ea7f12de56e446b19f407e6e092af4ba",
      "b1aba125897f4354a821bd5dd08ecd70",
      "f571d7cf478b4c20a1f5c4bd503e7737"
     ]
    },
    "id": "bRuvrZRfPIps",
    "outputId": "6207d868-c423-4df5-d1a0-05eb965be660"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "65fa5ad06c2841f4bf33c3d917dfe686",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/3081 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "67c086e6a2e347ce8378a7ea4041b27d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/771 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_dataset_float = train_dataset.map(lambda row: {'labels': [float(row['labels'])]})\n",
    "test_dataset_float = test_dataset.map(lambda row: {'labels': [float(row['labels'])]})\n",
    "\n",
    "train_tensor_dataset = tokenize_dataset(train_dataset_float, 'sentence', 'labels', auto_tokenizer, **tokenizer_kwargs)\n",
    "test_tensor_dataset = tokenize_dataset(test_dataset_float, 'sentence', 'labels', auto_tokenizer, **tokenizer_kwargs)\n",
    "\n",
    "generator = torch.Generator()\n",
    "train_loader = DataLoader(train_tensor_dataset, batch_size=4, shuffle=True, generator=generator)\n",
    "test_loader = DataLoader(test_tensor_dataset, batch_size=8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "a_zRWx9gPIps"
   },
   "source": [
    "### Model Configuration & Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 49,
     "referenced_widgets": [
      "7d9e3726eebd41f8b1d22aa80312df70",
      "11a777252a044e3da149877541e3ecaf",
      "e439a2c28d5c4ecf865619d7af82e698",
      "0b601c2d8959449ba0eedeb6e858a6b3",
      "9082dd5f5f2640a7a390a23684360ca9",
      "6fd602b801f0475db9c933fca5d2eb78",
      "852ae3ffc3f945d1b5fbf3dc377ffc07",
      "a1754c0bf4d9463ca5214eac4eb9a1eb",
      "106cc09c16f84c419d0d2cd5c7f55a8a",
      "98ecb94064cd4451b67cccdac5f8c102",
      "12ab9c1a8fac4be28462066d99d4cac5"
     ]
    },
    "id": "3Z1oEHNkPIps",
    "outputId": "806b04f1-81cf-45ba-e8ea-6a58a8ecb155"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7d9e3726eebd41f8b1d22aa80312df70",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model.safetensors:   0%|          | 0.00/268M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "torch.manual_seed(41)\n",
    "bert_model = AutoModel.from_pretrained(\"distilbert-base-uncased\")\n",
    "model = BERTClassifier(bert_model, 128, n_outputs=1)\n",
    "loss_fn = nn.BCEWithLogitsLoss()\n",
    "optimizer = optim.Adam(model.parameters(), lr=1e-5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {
    "id": "yZL4UQsVPIpt"
   },
   "outputs": [],
   "source": [
    "sbs_bert = StepByStep(model, loss_fn, optimizer)\n",
    "sbs_bert.set_loaders(train_loader, test_loader)\n",
    "sbs_bert.train(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4Uma227MPIpt",
    "outputId": "d4b503df-8d6a-4315-bf68-c4f4494f1f08"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "66461441"
      ]
     },
     "execution_count": 175,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sbs_bert.count_parameters()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "AOE-3A4jPIpt",
    "outputId": "3e1f3ecb-c726-4ff9-ac81-fc677ff1f3e0"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[446, 453],\n",
       "        [300, 318]])"
      ]
     },
     "execution_count": 176,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "StepByStep.loader_apply(test_loader, sbs_bert.correct)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "M-YD9bvDPIpt"
   },
   "source": [
    "# Fine-Tuning with HuggingFace"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BXihCcZSPIpt"
   },
   "source": [
    "## Sequence Classification (or Regression)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {
    "id": "VsEGVD3wPIpt"
   },
   "outputs": [],
   "source": [
    "from transformers import DistilBertForSequenceClassification\n",
    "torch.manual_seed(42)\n",
    "bert_cls = DistilBertForSequenceClassification.from_pretrained('distilbert-base-uncased', num_labels=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "w7MQ1ApTPIpu",
    "outputId": "6af547d2-b8ef-4912-ea74-b68ab6bd8896"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'transformers.models.distilbert.modeling_distilbert.DistilBertForSequenceClassification'>\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoModelForSequenceClassification\n",
    "auto_cls = AutoModelForSequenceClassification.from_pretrained('distilbert-base-uncased', num_labels=2)\n",
    "print(auto_cls.__class__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_3eWB1c2PIpu"
   },
   "source": [
    "## Tokenized Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {
    "id": "VmJOKQcmPIpu"
   },
   "outputs": [],
   "source": [
    "auto_tokenizer = AutoTokenizer.from_pretrained('distilbert-base-uncased')\n",
    "def tokenize(row):\n",
    "    return auto_tokenizer(row['sentence'],\n",
    "                          truncation=True,\n",
    "                          padding='max_length',\n",
    "                          max_length=30)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81,
     "referenced_widgets": [
      "d62f299a59d04f4da693b6524a501a2e",
      "4262c25b50d945fbbac58cb13f2cf4aa",
      "19f76eaac0a74742adc27e9cdad38b65",
      "608cdf8a656c4639ad7ed5bfbd77fab0",
      "e268bdded7af4cd39467d93be5b4621d",
      "42f3c23f79804b588e05bfddb0442b09",
      "ef3fc375222e4cf4a835fa898253cd8d",
      "eed9340ed9d04611b71f75e4db6cd67e",
      "bbb94e68c17e423f8dd9fc1a9438aa94",
      "44c46ae2a0dd486e8b12b5a35f8ff0fa",
      "65fa9425c1634aa7983701ae7a6b4be3",
      "98f94a4b641044809aa3bd2801fbee27",
      "c30bd63cc2694712a657bc50f69046e8",
      "aefcb034f9164e5ba707dd08c1e02a09",
      "6c9e2cc3d6b84d47bf71568457444be8",
      "33d1d6cd77c545d48c0cdc2a4207f05a",
      "6a07d0daadd84589abeca17d55fa0c8e",
      "938ab7319f314935afdab88c85d87894",
      "13192bc720d84c2cb83a4cca9906471a",
      "94ef455f4eb940c0b176827e8a7d9fc5",
      "da76d47c353f44068f5d2e41190e0d62",
      "4969585bd7e842038c8f4f7d3bc6efe1"
     ]
    },
    "id": "FzW2xAeDPIpu",
    "outputId": "b8f6aa5b-7e4c-46ad-e91a-733057bebbb2"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d62f299a59d04f4da693b6524a501a2e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/3081 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "98f94a4b641044809aa3bd2801fbee27",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/771 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tokenized_train_dataset = train_dataset.map(tokenize, batched=True)\n",
    "tokenized_test_dataset = test_dataset.map(tokenize, batched=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "MCToy9BpPIpv",
    "outputId": "9d635254-e995-4f5a-bfef-96d15525dc4b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'sentence': 'So the Woodman went at once to the trees and began to work; and he soon made a truck out of the limbs of trees, from which he chopped away all the leaves and branches.', 'source': 'wizoz10-1740.txt', 'labels': 0, 'input_ids': [101, 2061, 1996, 3536, 2386, 2253, 2012, 2320, 2000, 1996, 3628, 1998, 2211, 2000, 2147, 1025, 1998, 2002, 2574, 2081, 1037, 4744, 2041, 1997, 1996, 10726, 1997, 3628, 1010, 102], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}\n"
     ]
    }
   ],
   "source": [
    "print(tokenized_train_dataset[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {
    "id": "OJGHlBQ9PIpv"
   },
   "outputs": [],
   "source": [
    "tokenized_train_dataset.set_format(type='torch', columns=['input_ids', 'attention_mask', 'labels'])\n",
    "tokenized_test_dataset.set_format(type='torch', columns=['input_ids', 'attention_mask', 'labels'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "nsQJ3GjfPIpv",
    "outputId": "35bfafc2-bcc6-4a1a-8507-3b3cf54afdd2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'labels': tensor(0),\n",
       " 'input_ids': tensor([  101,  2061,  1996,  3536,  2386,  2253,  2012,  2320,  2000,  1996,\n",
       "          3628,  1998,  2211,  2000,  2147,  1025,  1998,  2002,  2574,  2081,\n",
       "          1037,  4744,  2041,  1997,  1996, 10726,  1997,  3628,  1010,   102]),\n",
       " 'attention_mask': tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "         1, 1, 1, 1, 1, 1])}"
      ]
     },
     "execution_count": 183,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenized_train_dataset[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pWBcmaN1PIpv"
   },
   "source": [
    "## Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {
    "id": "dHy5s29tPIpv"
   },
   "outputs": [],
   "source": [
    "from transformers import Trainer\n",
    "trainer = Trainer(model=bert_cls, train_dataset=tokenized_train_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "O7pDhbwdPIpw",
    "outputId": "906a36dc-a79d-422b-dc05-aaa87fedc467"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TrainingArguments(\n",
       "_n_gpu=1,\n",
       "accelerator_config={'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True},\n",
       "adafactor=False,\n",
       "adam_beta1=0.9,\n",
       "adam_beta2=0.999,\n",
       "adam_epsilon=1e-08,\n",
       "auto_find_batch_size=False,\n",
       "bf16=False,\n",
       "bf16_full_eval=False,\n",
       "data_seed=None,\n",
       "dataloader_drop_last=False,\n",
       "dataloader_num_workers=0,\n",
       "dataloader_persistent_workers=False,\n",
       "dataloader_pin_memory=True,\n",
       "dataloader_prefetch_factor=None,\n",
       "ddp_backend=None,\n",
       "ddp_broadcast_buffers=None,\n",
       "ddp_bucket_cap_mb=None,\n",
       "ddp_find_unused_parameters=None,\n",
       "ddp_timeout=1800,\n",
       "debug=[],\n",
       "deepspeed=None,\n",
       "disable_tqdm=True,\n",
       "dispatch_batches=None,\n",
       "do_eval=False,\n",
       "do_predict=False,\n",
       "do_train=False,\n",
       "eval_accumulation_steps=None,\n",
       "eval_delay=0,\n",
       "eval_steps=None,\n",
       "evaluation_strategy=no,\n",
       "fp16=False,\n",
       "fp16_backend=auto,\n",
       "fp16_full_eval=False,\n",
       "fp16_opt_level=O1,\n",
       "fsdp=[],\n",
       "fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False},\n",
       "fsdp_min_num_params=0,\n",
       "fsdp_transformer_layer_cls_to_wrap=None,\n",
       "full_determinism=False,\n",
       "gradient_accumulation_steps=1,\n",
       "gradient_checkpointing=False,\n",
       "gradient_checkpointing_kwargs=None,\n",
       "greater_is_better=None,\n",
       "group_by_length=False,\n",
       "half_precision_backend=auto,\n",
       "hub_always_push=False,\n",
       "hub_model_id=None,\n",
       "hub_private_repo=False,\n",
       "hub_strategy=every_save,\n",
       "hub_token=<HUB_TOKEN>,\n",
       "ignore_data_skip=False,\n",
       "include_inputs_for_metrics=False,\n",
       "include_num_input_tokens_seen=False,\n",
       "include_tokens_per_second=False,\n",
       "jit_mode_eval=False,\n",
       "label_names=None,\n",
       "label_smoothing_factor=0.0,\n",
       "learning_rate=5e-05,\n",
       "length_column_name=length,\n",
       "load_best_model_at_end=False,\n",
       "local_rank=0,\n",
       "log_level=passive,\n",
       "log_level_replica=warning,\n",
       "log_on_each_node=True,\n",
       "logging_dir=tmp_trainer/runs/Mar01_16-42-16_ca340f7611d1,\n",
       "logging_first_step=False,\n",
       "logging_nan_inf_filter=True,\n",
       "logging_steps=500,\n",
       "logging_strategy=steps,\n",
       "lr_scheduler_kwargs={},\n",
       "lr_scheduler_type=linear,\n",
       "max_grad_norm=1.0,\n",
       "max_steps=-1,\n",
       "metric_for_best_model=None,\n",
       "mp_parameters=,\n",
       "neftune_noise_alpha=None,\n",
       "no_cuda=False,\n",
       "num_train_epochs=3.0,\n",
       "optim=adamw_torch,\n",
       "optim_args=None,\n",
       "output_dir=tmp_trainer,\n",
       "overwrite_output_dir=False,\n",
       "past_index=-1,\n",
       "per_device_eval_batch_size=8,\n",
       "per_device_train_batch_size=8,\n",
       "prediction_loss_only=False,\n",
       "push_to_hub=False,\n",
       "push_to_hub_model_id=None,\n",
       "push_to_hub_organization=None,\n",
       "push_to_hub_token=<PUSH_TO_HUB_TOKEN>,\n",
       "ray_scope=last,\n",
       "remove_unused_columns=True,\n",
       "report_to=['tensorboard'],\n",
       "resume_from_checkpoint=None,\n",
       "run_name=tmp_trainer,\n",
       "save_on_each_node=False,\n",
       "save_only_model=False,\n",
       "save_safetensors=True,\n",
       "save_steps=500,\n",
       "save_strategy=steps,\n",
       "save_total_limit=None,\n",
       "seed=42,\n",
       "skip_memory_metrics=True,\n",
       "split_batches=None,\n",
       "tf32=None,\n",
       "torch_compile=False,\n",
       "torch_compile_backend=None,\n",
       "torch_compile_mode=None,\n",
       "torchdynamo=None,\n",
       "tpu_metrics_debug=False,\n",
       "tpu_num_cores=None,\n",
       "use_cpu=False,\n",
       "use_ipex=False,\n",
       "use_legacy_prediction_loop=False,\n",
       "use_mps_device=False,\n",
       "warmup_ratio=0.0,\n",
       "warmup_steps=0,\n",
       "weight_decay=0.0,\n",
       ")"
      ]
     },
     "execution_count": 185,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.args"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {
    "id": "vMVuTOSUPIpw"
   },
   "outputs": [],
   "source": [
    "from transformers import TrainingArguments\n",
    "training_args = TrainingArguments(\n",
    "    output_dir='output',\n",
    "    num_train_epochs=1,\n",
    "    per_device_train_batch_size=1,\n",
    "    per_device_eval_batch_size=8,\n",
    "    evaluation_strategy='steps',\n",
    "    eval_steps=300,\n",
    "    logging_steps=300,\n",
    "    gradient_accumulation_steps=8,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "metadata": {
    "id": "LFxIhoybPIpw"
   },
   "outputs": [],
   "source": [
    "def compute_metrics(eval_pred):\n",
    "    predictions = eval_pred.predictions\n",
    "    labels = eval_pred.label_ids\n",
    "    predictions = np.argmax(predictions, axis=1)\n",
    "    return {\"accuracy\": (predictions == labels).mean()}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {
    "id": "XKo42k7RPIpw"
   },
   "outputs": [],
   "source": [
    "trainer = Trainer(model=bert_cls,\n",
    "                  args=training_args,\n",
    "                  train_dataset=tokenized_train_dataset,\n",
    "                  eval_dataset=tokenized_test_dataset,\n",
    "                  compute_metrics=compute_metrics)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "9wdDiBP4PIpw",
    "outputId": "a92de2e2-5f72-4b2f-b8d5-cfa8f88965a2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 0.1904, 'grad_norm': 1.5486623048782349, 'learning_rate': 1.103896103896104e-05, 'epoch': 0.78}\n",
      "{'eval_loss': 0.12372920662164688, 'eval_accuracy': 0.9623865110246433, 'eval_runtime': 0.8077, 'eval_samples_per_second': 954.547, 'eval_steps_per_second': 120.092, 'epoch': 0.78}\n",
      "{'train_runtime': 70.4997, 'train_samples_per_second': 43.702, 'train_steps_per_second': 5.461, 'train_loss': 0.17047909823330967, 'epoch': 1.0}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TrainOutput(global_step=385, training_loss=0.17047909823330967, metrics={'train_runtime': 70.4997, 'train_samples_per_second': 43.702, 'train_steps_per_second': 5.461, 'train_loss': 0.17047909823330967, 'epoch': 1.0})"
      ]
     },
     "execution_count": 189,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.train()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "j_i6kvF5PIpx",
    "outputId": "aa3a7e0f-bf76-4c0d-9ea5-6af487251069"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.1257997453212738, 'eval_accuracy': 0.9623865110246433, 'eval_runtime': 0.9045, 'eval_samples_per_second': 852.392, 'eval_steps_per_second': 107.24, 'epoch': 1.0}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'eval_loss': 0.1257997453212738,\n",
       " 'eval_accuracy': 0.9623865110246433,\n",
       " 'eval_runtime': 0.9045,\n",
       " 'eval_samples_per_second': 852.392,\n",
       " 'eval_steps_per_second': 107.24,\n",
       " 'epoch': 1.0}"
      ]
     },
     "execution_count": 190,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.evaluate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2KOMV6iQPIpx",
    "outputId": "bf25f85c-fb6c-4be1-ad63-ea79faddc74f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['training_args.bin', 'model.safetensors', 'config.json']"
      ]
     },
     "execution_count": 191,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.save_model('bert_alice_vs_wizard')\n",
    "os.listdir('bert_alice_vs_wizard')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1WpIt7WvPIpx",
    "outputId": "9969675f-22ab-4603-e916-e32ea09cab49"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "device(type='cpu')"
      ]
     },
     "execution_count": 192,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loaded_model = AutoModelForSequenceClassification.from_pretrained('bert_alice_vs_wizard')\n",
    "loaded_model.device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "LrmSH5kSPIpx",
    "outputId": "36b1476a-3292-4b16-a4da-bbf9ae67487d",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "device(type='cuda', index=0)"
      ]
     },
     "execution_count": 193,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
    "loaded_model.to(device)\n",
    "loaded_model.device"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "I_6MnEPTPIpx"
   },
   "source": [
    "## Predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3_dt7C28PIpy",
    "outputId": "db608ee1-feab-4937-b9ef-1d372fdef591"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[  101,  2091,  1996,  3756,  5318, 10442,  4920,   102]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1]])}"
      ]
     },
     "execution_count": 194,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence = 'Down the yellow brick rabbit hole'\n",
    "tokens = auto_tokenizer(sentence, return_tensors='pt')\n",
    "tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "WAJtVv5fPIpy",
    "outputId": "5ff5314c-7043-495d-a81f-42a189388166"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'transformers.tokenization_utils_base.BatchEncoding'>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[  101,  2091,  1996,  3756,  5318, 10442,  4920,   102]],\n",
       "       device='cuda:0'), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1]], device='cuda:0')}"
      ]
     },
     "execution_count": 195,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(type(tokens))\n",
    "tokens.to(loaded_model.device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Fs4-3g2DPIpy",
    "outputId": "d4ef261b-86eb-48dc-e19a-25e3066248f4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SequenceClassifierOutput(loss=None, logits=tensor([[ 0.5956, -0.6552]], device='cuda:0', grad_fn=<AddmmBackward0>), hidden_states=None, attentions=None)"
      ]
     },
     "execution_count": 196,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loaded_model.eval()\n",
    "logits = loaded_model(input_ids=tokens['input_ids'], attention_mask=tokens['attention_mask'])\n",
    "logits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_81JJd68PIpy",
    "outputId": "46899912-3f24-41db-eec4-40fcc0ab49d3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0], device='cuda:0')"
      ]
     },
     "execution_count": 197,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logits.logits.argmax(dim=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AOIE2XY-PIpy"
   },
   "source": [
    "## Pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {
    "id": "eTdPgNWAPIpy"
   },
   "outputs": [],
   "source": [
    "from transformers import TextClassificationPipeline\n",
    "device_index = loaded_model.device.index if loaded_model.device.type != 'cpu' else -1\n",
    "classifier = TextClassificationPipeline(model=loaded_model,\n",
    "                                        tokenizer=auto_tokenizer,\n",
    "                                        device=device_index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rkhu6QaOPIpz",
    "outputId": "51be9f04-42df-40b8-fee4-f57bcc5d4273"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'label': 'LABEL_0', 'score': 0.7774350643157959},\n",
       " {'label': 'LABEL_1', 'score': 0.9984254837036133}]"
      ]
     },
     "execution_count": 199,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifier(['Down the Yellow Brick Rabbit Hole', 'Alice rules!'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {
    "id": "57PmBUrZPIpz"
   },
   "outputs": [],
   "source": [
    "loaded_model.config.id2label = {0: 'Wizard', 1: 'Alice'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 201,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2xsmzplHPIpz",
    "outputId": "ae42a9f6-0f05-47bd-fdb1-df680f0945d1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'label': 'Wizard', 'score': 0.7774350643157959},\n",
       " {'label': 'Alice', 'score': 0.9984254837036133}]"
      ]
     },
     "execution_count": 201,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifier(['Down the Yellow Brick Rabbit Hole', 'Alice rules!'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MPv9QGnPPIpz"
   },
   "source": [
    "## More Pipelines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 145,
     "referenced_widgets": [
      "fb82f5287b4a4a8c8e2ede1838c1dbeb",
      "6b7101cea911496e98007d220c75a05b",
      "57087b0cc5dd4a77be3beef53cb9a7d6",
      "7f8611121f774f5ea606cd968de34aa5",
      "e9164b241b124b83accf07de6272cf0b",
      "0c2b3e3f7a494b6bb3e46003c6770571",
      "3f6daa12dce64d348ddd971c5417726e",
      "b21725fa1e684f4c92418af03f389a6f",
      "e992d25a59fc4a71a9c1f02affe0e877",
      "4f792cc40ef34ae8a2b4a14d4538967e",
      "dc99ab2ea3704e0cb8279c142a738600",
      "1c7fb77282794a618a1b8175b77811aa",
      "edf1b0f7c7684e08a817590e9a201eb4",
      "0c699df3ee0e45008db6628bc9a6db67",
      "cb7fa84b89ca428f9004e14647b8c876",
      "9026426dbb884fe39e2ec321a89954b2",
      "efde0db3e6914b7f950b0a457bbcb4ae",
      "ee741717c13740faaf795193f91cbb6b",
      "78b58c720fdd4720aa4bee7197007309",
      "fce0f472357142ecaf627eeeee9c2e75",
      "ed9a7513f10e4bf390e853a725541922",
      "79b3244660d64718b33e95af4bbef1d5",
      "439d37b4dbec4ee4afc0167555253edb",
      "ea8505a10dee4b2f8e6d7ea9c3c30ed5",
      "002a9501cdfb408a84a8779cf0176b02",
      "62b8ba29be2c4518971e3ca4c03845c1",
      "1f15b8d5547441cbb156b8dfce4ec7b0",
      "35f6d77e4970448088eef39b7fbec3e0",
      "ddf9c083e48f415d92d2ce2322af149a",
      "4b81c0892898485fafa62fb76028a133",
      "6772d51b7bb544eaa1c86e0836e49766",
      "9f1735288ea745f39f95127366bb8698",
      "2a81d0cf6f654efeac3f2522b7fc0186",
      "4866c0b43a964ac8bf21db1a2c76852a",
      "c06eb99f4ec241b9b010660294f63bb2",
      "6b97eb69cc3d403abf0b32cd988f7706",
      "9f7521e21b0d4b918bdefd2eb38fa75a",
      "60635472c9f6460ab9d144e4c0a74271",
      "645f6ba5ef364830bfb4c43034233b18",
      "14e98cabb95c438abf13993167e4618f",
      "4cac570ecf964757aec9d3a24d86138c",
      "2ac98ce69f164fdfbfb7ff99ddbec967",
      "181ee900eccb4bdc98945cae0e95b770",
      "120fbeddc7d3406994260af25c636e5e"
     ]
    },
    "id": "orvV3n9wPIpz",
    "outputId": "152d9330-5ca4-49b7-81e9-9134b160eb05"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fb82f5287b4a4a8c8e2ede1838c1dbeb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "config.json:   0%|          | 0.00/629 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1c7fb77282794a618a1b8175b77811aa",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model.safetensors:   0%|          | 0.00/268M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "439d37b4dbec4ee4afc0167555253edb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer_config.json:   0%|          | 0.00/48.0 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4866c0b43a964ac8bf21db1a2c76852a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from transformers import pipeline\n",
    "sentiment = pipeline('sentiment-analysis')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ZGiwPvGfPIp0",
    "outputId": "0ab26a24-d4e6-4300-e06f-a679d6f02d23"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "So the Woodman went at once to the trees and began to work; and he soon made a truck out of the limbs of trees, from which he chopped away all the leaves and branches.\n",
      "[{'label': 'NEGATIVE', 'score': 0.9977670907974243}]\n"
     ]
    }
   ],
   "source": [
    "sentence = train_dataset[0]['sentence']\n",
    "print(sentence)\n",
    "print(sentiment(sentence))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 204,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "x456QWjEPIp0",
    "outputId": "33730915-95a4-421d-eb76-3e35240d6752"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'impl': transformers.pipelines.text_classification.TextClassificationPipeline,\n",
       " 'tf': (transformers.models.auto.modeling_tf_auto.TFAutoModelForSequenceClassification,),\n",
       " 'pt': (transformers.models.auto.modeling_auto.AutoModelForSequenceClassification,),\n",
       " 'default': {'model': {'pt': ('distilbert/distilbert-base-uncased-finetuned-sst-2-english',\n",
       "    'af0f99b'),\n",
       "   'tf': ('distilbert/distilbert-base-uncased-finetuned-sst-2-english',\n",
       "    'af0f99b')}},\n",
       " 'type': 'text'}"
      ]
     },
     "execution_count": 204,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from transformers.pipelines import SUPPORTED_TASKS\n",
    "# UPDATED\n",
    "###########################################################\n",
    "# sentiment-analysis was replaced by text-classification\n",
    "# in the dictionary of supported tasks\n",
    "# SUPPORTED_TASKS['sentiment-analysis']\n",
    "SUPPORTED_TASKS['text-classification']\n",
    "###########################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Xk3TWVXxPIp0",
    "outputId": "82f93a4b-b557-4c2e-9337-20a774ea0c71"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'impl': transformers.pipelines.text_generation.TextGenerationPipeline,\n",
       " 'tf': (transformers.models.auto.modeling_tf_auto.TFAutoModelForCausalLM,),\n",
       " 'pt': (transformers.models.auto.modeling_auto.AutoModelForCausalLM,),\n",
       " 'default': {'model': {'pt': ('openai-community/gpt2', '6c0e608'),\n",
       "   'tf': ('openai-community/gpt2', '6c0e608')}},\n",
       " 'type': 'text'}"
      ]
     },
     "execution_count": 205,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SUPPORTED_TASKS['text-generation']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qpfW8LWvPIp0"
   },
   "source": [
    "# GPT-2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 241,
     "referenced_widgets": [
      "0371f51c7d5d45b381014ff58b297ab1",
      "3e852e6825214df7bcda82aa091b55d1",
      "3b2ed30f07e844388dae47f81d9ea65b",
      "7fd547caf234435cb7952b211a538376",
      "3f3fadc89d2d4144a814b69d511a5dd7",
      "421997ecbc63432bbf12a049630dabab",
      "08ef888fa03449dfbfed2c38b82b7008",
      "960de534753949ada3cc727372f0db1d",
      "9e6171a639f544ceb56c091c57d9e619",
      "7742499693bb46a1888069358f2412a6",
      "5f9ee570b4034cfab50d5ce66f0f353f",
      "f4574e12d5db48ca862bfb2811fad258",
      "12afd487ae7343f7b8a65aaa60bd00c7",
      "d4ce47e5a2d34f4488171fd928918db3",
      "244018be291643fe9e929eaf3e531098",
      "f6e0e64e2c234c559c8160f06b28b618",
      "a7c1ffd3bfae4ac29997b77f805d6684",
      "42dcbe8766f74de69749e3e0600e3829",
      "4086656b8cc54030b47a881760d396e4",
      "b43bdb8595674ff9b9cd369c2edeea9f",
      "280a539281a34d13878b1d678e40a160",
      "89325c9398474f9c92f16c63156d3038",
      "979d607a3eee4671b0b93c5189bef09c",
      "e87270efd2594bbeb71d68f9f3f796f2",
      "e4f5bea0b35742499724bcfe092bbaba",
      "f0fefe6687614c32a11f90d91134f0e3",
      "8630987be068488bbca15d661a8ba777",
      "b036589813fb4676b956e89e846d4362",
      "687ec11348cd4044896fb407e3359a04",
      "c11c4c4acddf4a3cb62fcd72b50860d4",
      "3c290ca9b843494ea748504e4dd17655",
      "da814b1528be4926975893b54fd72849",
      "ce749881b13f4db38ccbc2d8a3fb09cd",
      "f9971e233de04afb9a9d4e6e2591d729",
      "925c24b98bec4b1ab62f4e2f3c5fc816",
      "ad3da69addc44dcd910c982aabd94fb5",
      "0bdfa1d6d98d42069dfd65dba4889c8e",
      "c61dc8ce463444d1909d23cd6071b355",
      "4665f471fa0e478882233b165c010b68",
      "53f0c26cedd14257988c3ea6cfb88cd0",
      "6d730e29f1794da08e4757a0bda3e67f",
      "622af824740148f8a404ce3cf428db53",
      "666266a80a094b599299c3dffd219c36",
      "29a292bd94c4456d8fd0eeb4973c0670",
      "9f3d7cd9902149938997025d735d83ab",
      "37f56d81b5664fbe933e6c31826a37e4",
      "2f488a3c842045d5ac7cc843f482427e",
      "b3e45aff192641afa35642453884a8e1",
      "a726e761ed344feb90b0058cf65498e7",
      "f12b26bf7ecb4040b23d9d8c2fff7a6e",
      "deb8982bab804d5c9fa3ba7627fad8d2",
      "b3735b82dae8447faaca2f82af9d39c6",
      "f244396283914d16baacaf5d6f76abac",
      "da491b6e5fde4debbfb30f5433848a60",
      "0d750e71cfd04203bc6245d72e62cdc5",
      "b53346bd908147038ebfb4b164580c10",
      "035cec73334d405d860369ff47340d4d",
      "c55f4a29cd6b46e285f2b3b26e34ff17",
      "5923d06569814e5ba74e490eaf5c3c37",
      "804f844063e24173baf5e34af9cad49c",
      "8bd98770a54b40eba0468d1c7682c308",
      "27a4e299f86a47f6a77cb2fde6fd9607",
      "d5cc7296b9714a448e44384bf8f62bf3",
      "0392f7a3461842f3bc9ef9be05242229",
      "dbcdf8b2ea64486a8fb9b93aa29c546a",
      "5ecabaa3f4904b52b9bfc6933ac4cd90",
      "d847ba4af98546978dafacf9e9b0fc95",
      "99b052cccf804bc3bbecb3d2a7b8eebb",
      "bd1e564382c4439c82645445d421ecbf",
      "6f6b8a2799d9496a820168a6a33a62d5",
      "7e32b26c293e45b18eaa2e0c96ab145c",
      "fcdff5e01eb7414c9325064a9be4f92a",
      "ad5a96afd3bd478e9bb9e412d2770c9e",
      "8d9d4561ad3b4a6f865dadcfda4ac72a",
      "8f34f724a32e486da536cf0e55f3173c",
      "a338d6cd66b144819521d0ab553719f1",
      "9911f6d5447a410b9bf7a0e3dc966b13"
     ]
    },
    "id": "0Gs2m5QtPIp0",
    "outputId": "6f757279-0659-4586-ff57-6d18dc7156ac"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0371f51c7d5d45b381014ff58b297ab1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "config.json:   0%|          | 0.00/665 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f4574e12d5db48ca862bfb2811fad258",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model.safetensors:   0%|          | 0.00/548M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "979d607a3eee4671b0b93c5189bef09c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "generation_config.json:   0%|          | 0.00/124 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f9971e233de04afb9a9d4e6e2591d729",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer_config.json:   0%|          | 0.00/26.0 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9f3d7cd9902149938997025d735d83ab",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "vocab.json:   0%|          | 0.00/1.04M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b53346bd908147038ebfb4b164580c10",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d847ba4af98546978dafacf9e9b0fc95",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer.json:   0%|          | 0.00/1.36M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "text_generator = pipeline(\"text-generation\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "sNIhhyloPIp1",
    "outputId": "3a2e584c-5c3a-48ab-edeb-2a8069dde358"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'text-generation': {'do_sample': True, 'max_length': 50}}"
      ]
     },
     "execution_count": 208,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_generator.model.config.task_specific_params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 209,
   "metadata": {
    "id": "JAlA7qvJPIp1"
   },
   "outputs": [],
   "source": [
    "base_text = \"\"\"\n",
    "Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do:  once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, `and what is the use of a book,'thought Alice `without pictures or conversation?' So she was considering in her own mind (as well as she could, for the hot day made her feel very sleepy and stupid), whether the pleasure of making a daisy-chain would be worth the trouble of getting up and picking the daisies, when suddenly a White Rabbit with pink eyes ran close by her.\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "OGsxW7xVPIp1",
    "outputId": "4343f0c4-b6aa-4a7a-8a46-f0eb28a91439"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do:  once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, `and what is the use of a book,'thought Alice `without pictures or conversation?' So she was considering in her own mind (as well as she could, for the hot day made her feel very sleepy and stupid), whether the pleasure of making a daisy-chain would be worth the trouble of getting up and picking the daisies, when suddenly a White Rabbit with pink eyes ran close by her.\n",
      "\n",
      "'What did you say?'\n",
      "\n",
      "No way. She was beginning to wonder: 'Well, what happened?'\n",
      "\n",
      "'Well this isn't your fault. The White Rabbit was about to run past that thing over there. And, since it's too long, he was only standing there to stare.'\n",
      "\n",
      "'That's your fault,' said Alice `Well, we must remember we've got an old boy to check the house. We'll know what happens if he runs over our bed and we find the room. What do you think?'\n"
     ]
    }
   ],
   "source": [
    "result = text_generator(base_text, max_length=250)\n",
    "print(result[0]['generated_text'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Kb1uve8LPIp1"
   },
   "source": [
    "# Putting It All Together"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BBjmamx_PIp1"
   },
   "source": [
    "## Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 49,
     "referenced_widgets": [
      "f2a9bf4b0fec4fb6a0942174e2198046",
      "ec120f1e3ebd44f1bc98d8fd7bfecc14",
      "9ba88a2e42b54a59a6aa9907e5c57fa0",
      "3b0e267772fa48fa87c6cc767885a590",
      "3e08a44cdaee4ec18a767c85d61e89cb",
      "86b098ea173a4419a9fe67d9c165b95b",
      "02c2e1530b63441e9954920f3c27a606",
      "8a2d5a4ee0fd473b8137b518bfc3d68b",
      "67d215dd5e024d0d922c6ea2a8a1b36f",
      "a647eaa49402408b8b10d8efb4c19bfd",
      "4e505ed8ff2d4b62b139dbc7631f83bd"
     ]
    },
    "id": "rKf7qtJkPIp1",
    "outputId": "18ea1b4f-d4b8-493a-895a-625a8b8bb4ff"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f2a9bf4b0fec4fb6a0942174e2198046",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating train split: 0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dataset = load_dataset(path='csv', data_files=['texts/alice28-1476.sent.csv'], quotechar='\\\\', split=Split.TRAIN)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "metadata": {
    "id": "sRxp47A-PIp2"
   },
   "outputs": [],
   "source": [
    "shuffled_dataset = dataset.shuffle(seed=42)\n",
    "split_dataset = shuffled_dataset.train_test_split(test_size=0.2, seed=42)\n",
    "train_dataset, test_dataset = split_dataset['train'], split_dataset['test']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 177,
     "referenced_widgets": [
      "20f3a0f123d34cc28b5b4b69a350ea8d",
      "ff804399b8e64b4ea6795b5df7ac99ea",
      "88445353976648f583ff1ec689f04d09",
      "690943997d60421d8b948a0e0bf1e957",
      "61d08e8218eb4c37a97a27ba42333517",
      "1dff6d902ca749d29775a39b53e98e03",
      "f21888da6c744c68994334969d6656cb",
      "7b53e5a3ecba425194221dab9273f08f",
      "b6c7ff63249546808799c3b45711ede5",
      "ca388e55eafd41a483980574fb090a22",
      "b816a4edc3bd41e0868a82e1d007e067",
      "5dec607d9f0847c3be255d78656ef52b",
      "fae5f821c5604402af8f68f46ed4f2ca",
      "5929f3be61e64cd3b03db83c6315f113",
      "a8d497b06b8648fe8c06905fe9ebc973",
      "a575be776efd4fa2be2bb978262efcfa",
      "739a81b3b3d44ac9b906227cad969bbc",
      "d3a7c090b74f43c7bc2b6008ec0f44ae",
      "09e572cc5fa34a3dbfa76f898c90edab",
      "c4381a91790446799969ce55f8a7250c",
      "7c460ab96eed403cadba9519916c112b",
      "ba8a10a3495d4a5b87e65562d115f7c5",
      "5bf02116b4ae46389f0d60bf7f7b8816",
      "77c90f063ce747aa9db15f2ae48d839d",
      "9533899297ed48d1aa92583084412d9a",
      "6b9430e328e246978f9162e256060cec",
      "7e3f568ed5f5440e93379c6d5eb85a4b",
      "cfbaf259829247f49994bf1e605b3e03",
      "60e6df2744cf40afaee1cbe071eae01e",
      "85b192763f8a4480a3d4ae4e0957b685",
      "a186cd0d62634611941bbe875d707ab3",
      "29d95cc429b74415b08672657ab320c7",
      "86875d1a86354e4787e872c98083210d",
      "196b861291ea451cbae8640523878913",
      "47cfdf980592498b893f8c3d2fbf0e03",
      "db17d88724f845568a4396b9a5ce508a",
      "adaaaa75b6f047239329f6a74b7c2f77",
      "784f4fb1e01241adab3acfcbf341a0ec",
      "8cf83ceaa38044578713806efb84870a",
      "311e3440f7a74343af959514ab41f35c",
      "eb8c530d239d4a4bb39f0296e67dc76d",
      "510e63d365c547b1adaff42d00490401",
      "3133811b808e4fdfbcdaf81abb3a994e",
      "0775a4d8fe1d4cd3b34486bf28d5f755",
      "107b8dc81c084768b72361170b3b448a",
      "e4db92d02bbd416daed51e431353de58",
      "9ad0006be39642f69cd73ee3bd5161d2",
      "99118a9b59864352ad3870d12b233d80",
      "db1d3dc2171b4910a5e07ba1e7d5f734",
      "be8fcd3bd159483081ab057dcc861aa2",
      "397a0d9b88b2483ab2736037772e97e8",
      "4e10af0f5cec42fb8cb3247af6430909",
      "4edd5b00fbdb41ca8347c9884506837b",
      "fbdf116a3aec45499f53c40a76b95741",
      "af134db80bf741949c6c7248d921e874"
     ]
    },
    "id": "gVF5IzXsPIp2",
    "outputId": "3255dc85-d5ae-4244-c501-8ab57bd39068"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "20f3a0f123d34cc28b5b4b69a350ea8d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer_config.json:   0%|          | 0.00/26.0 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5dec607d9f0847c3be255d78656ef52b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "config.json:   0%|          | 0.00/665 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5bf02116b4ae46389f0d60bf7f7b8816",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "vocab.json:   0%|          | 0.00/1.04M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "196b861291ea451cbae8640523878913",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "107b8dc81c084768b72361170b3b448a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer.json:   0%|          | 0.00/1.36M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "auto_tokenizer = AutoTokenizer.from_pretrained('gpt2')\n",
    "def tokenize(row):\n",
    "    return auto_tokenizer(row['sentence'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81,
     "referenced_widgets": [
      "6d2a23d441284fc58af4502bd9c213f5",
      "4d8e31a1dc0c4787891c15c6036ffab1",
      "e1b5654d0d294ef9a0ac65bad7e41ad6",
      "da6b104c271d4ea387af13b328e40fb6",
      "2daec0b4c2c941d2a2be950111102b2d",
      "5e020a3384894ce9adffb1198a2eb181",
      "5797df4963724a82821caf8d0b662274",
      "8b1d3a21ee4b4808847ea2a82a18ac61",
      "57c90e7dadbe4879a2baaf4f22722f37",
      "8df7d56b3bb745caa64370e4c60e4205",
      "5bf3f7314dc74045aaa4cc1c4cd7bc7d",
      "855ca149e747497f8a126585fe6abda7",
      "a5fe3f2998b44c4981f31f6c3070d7df",
      "78f879c46a7f4a18ae2de70654121f0f",
      "3871c4f951d94bd1a86dc61312e3b6d0",
      "f99199d9122f49a0a36611c7e2e76419",
      "a747ee6a682d4888a16451b2fe57967d",
      "b47d7ef257c449f1b0e8756446997d89",
      "01d0e06565c841cf8462f6d041873ad8",
      "bf35bcd4037b4cfaa8554c210951ad29",
      "df66efedce50453f83d29acdd5f8b10e",
      "441949f9bf114b879fb153334abb910b"
     ]
    },
    "id": "YUBOlXsqPIp2",
    "outputId": "db778bc2-4c38-4284-bca1-841c58e2d061"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6d2a23d441284fc58af4502bd9c213f5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/1289 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "855ca149e747497f8a126585fe6abda7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/323 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tokenized_train_dataset = train_dataset.map(tokenize, remove_columns=['source', 'sentence'], batched=True)\n",
    "tokenized_test_dataset = test_dataset.map(tokenize, remove_columns=['source', 'sentence'], batched=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "m-4cJ4pAPIp2",
    "outputId": "111a91b4-7ec1-4b29-82bb-821063020c7b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[9, 28, 20, 9, 34, 29]"
      ]
     },
     "execution_count": 215,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(map(len, tokenized_train_dataset[0:6]['input_ids']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VKWqqWJ3PIp2"
   },
   "source": [
    "### \"Packed\" Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "U7aJwvXsPIp3"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/block_tokens.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 216,
   "metadata": {
    "id": "7lAMC3OtPIp3"
   },
   "outputs": [],
   "source": [
    "# Adapted from https://github.com/huggingface/transformers/blob/master/examples/pytorch/language-modeling/run_clm.py\n",
    "def group_texts(examples, block_size=128):\n",
    "    # Concatenate all texts.\n",
    "    concatenated_examples = {k: sum(examples[k], []) for k in examples.keys()}\n",
    "    total_length = len(concatenated_examples[list(examples.keys())[0]])\n",
    "    # We drop the small remainder, we could add padding if the model supported it instead of this drop, you can\n",
    "    # customize this part to your needs.\n",
    "    total_length = (total_length // block_size) * block_size\n",
    "    # Split by chunks of max_len.\n",
    "    result = {\n",
    "        k: [t[i : i + block_size] for i in range(0, total_length, block_size)]\n",
    "        for k, t in concatenated_examples.items()\n",
    "    }\n",
    "    result[\"labels\"] = result[\"input_ids\"].copy()\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81,
     "referenced_widgets": [
      "9ec4bf7395dc42ea8022959062f6f333",
      "bfdf1f0d388942beb50bcd7242bdb3df",
      "34ea08329bce40839467041d8d675b0b",
      "9ef31f7bcb7342108ac855ceb0d76b49",
      "f2c83ee98d4d44389e9354b89a914cd1",
      "d2a05abfd84e402d811f9a976d724478",
      "03ffc521b0874dc6aeb0beb2cefae358",
      "937d3dea8a2547f38287f8c61c8a3b8e",
      "5e9280d23a81451da9d5898d9f5684b6",
      "ea6dca3ac29d409daa542899e8b7a3e1",
      "8bfbfd341cc54418aaa9ca300aa6f54a",
      "33268ee44c654b9e9154dae67b432ddc",
      "e9fe5354ad2749b6a3bb9d301a883f29",
      "f145daed41fc44b3a65c58b81470c919",
      "6c8aa9cbdfa64cba90c1c3ee782cf348",
      "152381e5fe68427393a8a334a98e14a5",
      "60784bb139dd474b86f3bd48c1a3103f",
      "0e366bd39b2e4bb78bf145447a329555",
      "eb0b856aedb744b9831994bdd921d705",
      "65a0252124e24f13a7db5e4c1bf29fc2",
      "cdf02697fb654a65bcdf5886f102b4da",
      "81304ddd745d4ed5be9b97220aa86095"
     ]
    },
    "id": "TirMA67-PIp3",
    "outputId": "a5e22cc0-3bfe-473c-9ebb-a781d3c5b466"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9ec4bf7395dc42ea8022959062f6f333",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/1289 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "33268ee44c654b9e9154dae67b432ddc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/323 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "lm_train_dataset = tokenized_train_dataset.map(group_texts, batched=True)\n",
    "lm_test_dataset = tokenized_test_dataset.map(group_texts, batched=True)\n",
    "lm_train_dataset.set_format(type='torch')\n",
    "lm_test_dataset.set_format(type='torch')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 218,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lwOa4nluPIp3",
    "outputId": "ceffc745-51d8-4b97-9724-15ba02a3c4b6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([   63,  2437,   466,   345,   760,   314,  1101,  8805,  8348,   464,\n",
      "         2677,  3114,  7296,  6819,   379,   262,  2635, 25498,    11,   508,\n",
      "          531,   287,   257,  1877,  3809,    11,  4600,  7120, 25788,  1276,\n",
      "         3272,    12,  1069,  9862, 12680,  4973,  2637,  1537,   611,   314,\n",
      "         1101,   407,   262,   976,    11,   262,  1306,  1808,   318,    11,\n",
      "         5338,   287,   262,   995,   716,   314,    30,   464,   360,   579,\n",
      "         1076,  6364,  4721,   465,  2951,    13,    63,  1026,   373,   881,\n",
      "        21289,   272,   353,   379,  1363,  4032,  1807,  3595, 14862,    11,\n",
      "         4600, 12518,   530,  2492,   470,  1464,  3957,  4025,   290,  4833,\n",
      "           11,   290,   852,  6149,   546,   416, 10693,   290, 33043,    13,\n",
      "         1870, 14862,   373,   523,   881, 24776,   326,   673,  4966,   572,\n",
      "          379,  1752,   287,   262,  4571,   340,  6235,   284,    11,  1231,\n",
      "         2111,   284,  4727,   262,  7457,   340,   550,   925])\n"
     ]
    }
   ],
   "source": [
    "print(lm_train_dataset[0]['input_ids'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hkJM5tc3PIp3",
    "outputId": "54dafc25-945c-4fe6-f425-354b24b10e27"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(239, 56)"
      ]
     },
     "execution_count": 219,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(lm_train_dataset), len(lm_test_dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MgLxoOb-PIp3"
   },
   "source": [
    "## Model Configuration & Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 220,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 99,
     "referenced_widgets": [
      "47989a32d3e54ede913cad2e881407ba",
      "7273d05e25dd4af98ca58416bc2d1cb5",
      "271083e34adf45bb8aab6b1e85193706",
      "e5786c3017bf4a639a11f54f3aa7d9ce",
      "51e8a4fe908c4a5b8895469ecfbe631b",
      "b2414b78400c4a4c8a87cbbed9668d13",
      "3560a5a7dea44f0fb8b56a5f83fd5597",
      "e5109813dc714864851c06b46db9cae9",
      "8234a4f00aef440c8ebd2069ff421853",
      "cd47ce9df8cf4d4aae89672fc0602b71",
      "d0786942e0df48e78edc03829aa53a0c",
      "4a7dc41aa2824f918493115bda24498b",
      "387e47830d614fe8ba6758c7485a820c",
      "85773884b394433b8f47703742a0413a",
      "83f97c79b32d40f18c79d29974790dcf",
      "340765c79f7b4b1fa7dc51f0db682e9c",
      "8d567116324241c88d40d887ba44bbd1",
      "edfecdad72b345c2ab68a674e3aee9ff",
      "55dff7016e7a437ab8a732518f65ca4c",
      "4de0dcc6dfdb436abf0769cb0b2cf6e2",
      "b00cda330cfb40a9a35127f2cadf72d4",
      "10a85b00fe6740bc8611137ac5bfc137"
     ]
    },
    "id": "q9NLtw1APIp4",
    "outputId": "db7945f3-1553-4328-c58f-31db9f689a6a"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "47989a32d3e54ede913cad2e881407ba",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model.safetensors:   0%|          | 0.00/548M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4a7dc41aa2824f918493115bda24498b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "generation_config.json:   0%|          | 0.00/124 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'transformers.models.gpt2.modeling_gpt2.GPT2LMHeadModel'>\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoModelForCausalLM\n",
    "model = AutoModelForCausalLM.from_pretrained('gpt2')\n",
    "print(model.__class__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 221,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4fI5PE_nPIp4",
    "outputId": "a1fbb612-fa24-4524-8c2c-8c45dce661b2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Embedding(50257, 768)"
      ]
     },
     "execution_count": 221,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.resize_token_embeddings(len(auto_tokenizer))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 222,
   "metadata": {
    "id": "Ta_7BbrVPIp4"
   },
   "outputs": [],
   "source": [
    "training_args = TrainingArguments(\n",
    "    output_dir='output',\n",
    "    num_train_epochs=1,\n",
    "    per_device_train_batch_size=1,\n",
    "    per_device_eval_batch_size=8,\n",
    "    evaluation_strategy='steps',\n",
    "    eval_steps=50,\n",
    "    logging_steps=50,\n",
    "    gradient_accumulation_steps=4,\n",
    "    prediction_loss_only=True,\n",
    ")\n",
    "\n",
    "trainer = Trainer(model=model,\n",
    "                  args=training_args,\n",
    "                  train_dataset=lm_train_dataset,\n",
    "                  eval_dataset=lm_test_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 223,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3WQN-YhePIp4",
    "outputId": "856f062a-073d-4ada-ca21-f6c7fca4d091"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 3.567, 'grad_norm': 11.076814651489258, 'learning_rate': 7.627118644067798e-06, 'epoch': 0.84}\n",
      "{'eval_loss': 3.336221933364868, 'eval_runtime': 0.5964, 'eval_samples_per_second': 93.899, 'eval_steps_per_second': 11.737, 'epoch': 0.84}\n",
      "{'train_runtime': 14.5125, 'train_samples_per_second': 16.469, 'train_steps_per_second': 4.065, 'train_loss': 3.5485167745816506, 'epoch': 0.99}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TrainOutput(global_step=59, training_loss=3.5485167745816506, metrics={'train_runtime': 14.5125, 'train_samples_per_second': 16.469, 'train_steps_per_second': 4.065, 'train_loss': 3.5485167745816506, 'epoch': 0.99})"
      ]
     },
     "execution_count": 223,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.train()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 224,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_ru9UY-NPIp5",
    "outputId": "d1acb1c1-74a4-43cb-a08f-4bcad4407226"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 3.329355478286743, 'eval_runtime': 0.5829, 'eval_samples_per_second': 96.075, 'eval_steps_per_second': 12.009, 'epoch': 0.99}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'eval_loss': 3.329355478286743,\n",
       " 'eval_runtime': 0.5829,\n",
       " 'eval_samples_per_second': 96.075,\n",
       " 'eval_steps_per_second': 12.009,\n",
       " 'epoch': 0.99}"
      ]
     },
     "execution_count": 224,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.evaluate()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UYsl0aOiPIp5"
   },
   "source": [
    "## Generating Text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 225,
   "metadata": {
    "id": "xhkK5TIFPIp5"
   },
   "outputs": [],
   "source": [
    "device_index = model.device.index if model.device.type != 'cpu' else -1\n",
    "gpt2_gen = pipeline('text-generation', model=model, tokenizer=auto_tokenizer, device=device_index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 226,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PF0RfSp4PIp5",
    "outputId": "68cbacb4-2669-4649-f6d7-f0488ae1dc9e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do:  once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, `and what is the use of a book,'thought Alice `without pictures or conversation?' So she was considering in her own mind (as well as she could, for the hot day made her feel very sleepy and stupid), whether the pleasure of making a daisy-chain would be worth the trouble of getting up and picking the daisies, when suddenly a White Rabbit with pink eyes ran close by her.\n",
      "                          \n",
      " Alice watched one of the birds on the other side of their nest, where it was the only one in sight.        The other was sitting there, with its little mouth gaping asunder:                                            (\n"
     ]
    }
   ],
   "source": [
    "result = gpt2_gen(base_text, max_length=250)\n",
    "print(result[0]['generated_text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "DQG_yBYhPIp6"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "gpuType": "T4",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "002a9501cdfb408a84a8779cf0176b02": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4b81c0892898485fafa62fb76028a133",
      "max": 48,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_6772d51b7bb544eaa1c86e0836e49766",
      "value": 48
     }
    },
    "01d0e06565c841cf8462f6d041873ad8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "01ddfefb7f3e4a3589efab6d307a1bf0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "02c2e1530b63441e9954920f3c27a606": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "035cec73334d405d860369ff47340d4d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8bd98770a54b40eba0468d1c7682c308",
      "placeholder": "​",
      "style": "IPY_MODEL_27a4e299f86a47f6a77cb2fde6fd9607",
      "value": "merges.txt: 100%"
     }
    },
    "0371f51c7d5d45b381014ff58b297ab1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_3e852e6825214df7bcda82aa091b55d1",
       "IPY_MODEL_3b2ed30f07e844388dae47f81d9ea65b",
       "IPY_MODEL_7fd547caf234435cb7952b211a538376"
      ],
      "layout": "IPY_MODEL_3f3fadc89d2d4144a814b69d511a5dd7"
     }
    },
    "0392f7a3461842f3bc9ef9be05242229": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "03b0d4b80ee8400e8bb1ff96fe794139": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5a124cf7785e4eccafa4d1c7d3cd2942",
      "max": 483,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_1809cd29e041406687b13786774d4880",
      "value": 483
     }
    },
    "03ffc521b0874dc6aeb0beb2cefae358": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "046484ba77494b55acb7e13ed55a3487": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b2610c7ea73e437083cd43ef671cc563",
      "max": 48,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_514fd68e776d4e4aaeb4a90f51bc4736",
      "value": 48
     }
    },
    "063a969f176941fa84a21569b4c6ba2b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "072fea3dc9ba40f0b6e781f8755a322f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0775a4d8fe1d4cd3b34486bf28d5f755": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "08ef888fa03449dfbfed2c38b82b7008": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "090e8bfcf7ed4be6870d9fcb068100a3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_dc87398fce094a658da749f69feb4fc8",
       "IPY_MODEL_4e07a852aa1b4cf1bb57b7c77515c133",
       "IPY_MODEL_8e8a43aa4e11455484df7d141e6ad248"
      ],
      "layout": "IPY_MODEL_3c644b03c6ca4f388efbfe3f9cd573a4"
     }
    },
    "09e572cc5fa34a3dbfa76f898c90edab": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0b601c2d8959449ba0eedeb6e858a6b3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_98ecb94064cd4451b67cccdac5f8c102",
      "placeholder": "​",
      "style": "IPY_MODEL_12ab9c1a8fac4be28462066d99d4cac5",
      "value": " 268M/268M [00:00&lt;00:00, 337MB/s]"
     }
    },
    "0bdfa1d6d98d42069dfd65dba4889c8e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_666266a80a094b599299c3dffd219c36",
      "placeholder": "​",
      "style": "IPY_MODEL_29a292bd94c4456d8fd0eeb4973c0670",
      "value": " 26.0/26.0 [00:00&lt;00:00, 969B/s]"
     }
    },
    "0c2b3e3f7a494b6bb3e46003c6770571": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0c699df3ee0e45008db6628bc9a6db67": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_78b58c720fdd4720aa4bee7197007309",
      "max": 267832558,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_fce0f472357142ecaf627eeeee9c2e75",
      "value": 267832558
     }
    },
    "0c78bace44154985ad12c0265ae430f7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0d25366a3d654d3fba85508bf309e568": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "0d27001bb0cb4b689a75fbea7f1efa4c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0d750e71cfd04203bc6245d72e62cdc5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0db028ff95e84d33acdb533760ad41da": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0dcaee584779404aa4fbbad506ddcd3a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_53a7239ac99e4fd889edf4be66091d35",
       "IPY_MODEL_57b9fbf00fe34967b6f342524aae352b",
       "IPY_MODEL_e6f9a976e59e42618a3c511f427acb88"
      ],
      "layout": "IPY_MODEL_7ce5227fb27046dfb9f47c48aa863fb8"
     }
    },
    "0e0863081114481c964e57672c86c0b7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0e366bd39b2e4bb78bf145447a329555": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "10570d170bec405c8c15d80523a0717c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "106cc09c16f84c419d0d2cd5c7f55a8a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "107b8dc81c084768b72361170b3b448a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_e4db92d02bbd416daed51e431353de58",
       "IPY_MODEL_9ad0006be39642f69cd73ee3bd5161d2",
       "IPY_MODEL_99118a9b59864352ad3870d12b233d80"
      ],
      "layout": "IPY_MODEL_db1d3dc2171b4910a5e07ba1e7d5f734"
     }
    },
    "10a85b00fe6740bc8611137ac5bfc137": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "11a777252a044e3da149877541e3ecaf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6fd602b801f0475db9c933fca5d2eb78",
      "placeholder": "​",
      "style": "IPY_MODEL_852ae3ffc3f945d1b5fbf3dc377ffc07",
      "value": "model.safetensors: 100%"
     }
    },
    "120b3e46872b41cdb2be157882b67ca4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d9cdf9667f6746f382f5f10f772b3144",
      "max": 231508,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5e05b1fd6b084563bc6b22e5650c6c79",
      "value": 231508
     }
    },
    "120fbeddc7d3406994260af25c636e5e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "12ab9c1a8fac4be28462066d99d4cac5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "12afd487ae7343f7b8a65aaa60bd00c7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a7c1ffd3bfae4ac29997b77f805d6684",
      "placeholder": "​",
      "style": "IPY_MODEL_42dcbe8766f74de69749e3e0600e3829",
      "value": "model.safetensors: 100%"
     }
    },
    "12e1763b7a9b4ae78074cee4b5a409a3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f601a3994de84cc9885036e8d1c5a8ee",
      "placeholder": "​",
      "style": "IPY_MODEL_64cecc3cdf97483990e52c2b0311601f",
      "value": " 48.0/48.0 [00:00&lt;00:00, 3.53kB/s]"
     }
    },
    "13192bc720d84c2cb83a4cca9906471a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "13a2a27f04f7451aa8be99bb139efb27": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f28cbac2858e4e079790d7f24ff1d73d",
      "max": 466062,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_975c4167faed4d339bbc3f2eeae9f748",
      "value": 466062
     }
    },
    "14d54788f46d479eb0faa38af384cdc8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "14e98cabb95c438abf13993167e4618f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "152381e5fe68427393a8a334a98e14a5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1612ba093f8c455fb8149092cf366571": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "16514425790a4386ba23ea1ff1cdda3b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "1809cd29e041406687b13786774d4880": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "181ee900eccb4bdc98945cae0e95b770": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "196b861291ea451cbae8640523878913": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_47cfdf980592498b893f8c3d2fbf0e03",
       "IPY_MODEL_db17d88724f845568a4396b9a5ce508a",
       "IPY_MODEL_adaaaa75b6f047239329f6a74b7c2f77"
      ],
      "layout": "IPY_MODEL_784f4fb1e01241adab3acfcbf341a0ec"
     }
    },
    "19a2336a406a45bf9e990fa8ad9a49d4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_44050ee5b91145a8b22e2bc7de3ba85a",
      "placeholder": "​",
      "style": "IPY_MODEL_2db54785b3554d46811fde033a15dc17",
      "value": "tokenizer.json: 100%"
     }
    },
    "19f76eaac0a74742adc27e9cdad38b65": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_eed9340ed9d04611b71f75e4db6cd67e",
      "max": 3081,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_bbb94e68c17e423f8dd9fc1a9438aa94",
      "value": 3081
     }
    },
    "1bc31bdb12804834aa1e94ee7fa1b3ab": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b7d267442de04961a6f52c38d00c2edc",
      "placeholder": "​",
      "style": "IPY_MODEL_ebacdfc661154ae386dd3ccaeb23f772",
      "value": " 3081/3081 [00:35&lt;00:00, 57.90 examples/s]"
     }
    },
    "1c7fb77282794a618a1b8175b77811aa": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_edf1b0f7c7684e08a817590e9a201eb4",
       "IPY_MODEL_0c699df3ee0e45008db6628bc9a6db67",
       "IPY_MODEL_cb7fa84b89ca428f9004e14647b8c876"
      ],
      "layout": "IPY_MODEL_9026426dbb884fe39e2ec321a89954b2"
     }
    },
    "1dff6d902ca749d29775a39b53e98e03": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1ea9534a1322429eba100f7a28b3cbb2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_70db9c1a40214b50a30862988d18e717",
      "max": 440449768,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_e3941149c2a84b6ab6f61b6666e393af",
      "value": 440449768
     }
    },
    "1f15b8d5547441cbb156b8dfce4ec7b0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1f95a9be7f54444b88744c67e8da5a80": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "208deb72133c40a7a88e01add562fd60": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9fcc4a2ca21042848e5becbb152cb681",
      "placeholder": "​",
      "style": "IPY_MODEL_1612ba093f8c455fb8149092cf366571",
      "value": "model.safetensors: 100%"
     }
    },
    "20f3a0f123d34cc28b5b4b69a350ea8d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ff804399b8e64b4ea6795b5df7ac99ea",
       "IPY_MODEL_88445353976648f583ff1ec689f04d09",
       "IPY_MODEL_690943997d60421d8b948a0e0bf1e957"
      ],
      "layout": "IPY_MODEL_61d08e8218eb4c37a97a27ba42333517"
     }
    },
    "244018be291643fe9e929eaf3e531098": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_280a539281a34d13878b1d678e40a160",
      "placeholder": "​",
      "style": "IPY_MODEL_89325c9398474f9c92f16c63156d3038",
      "value": " 548M/548M [00:06&lt;00:00, 164MB/s]"
     }
    },
    "2493c820244c4097a40125268920c512": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_208deb72133c40a7a88e01add562fd60",
       "IPY_MODEL_1ea9534a1322429eba100f7a28b3cbb2",
       "IPY_MODEL_61d1ee6617f3473da9b5e40b1ac00fbf"
      ],
      "layout": "IPY_MODEL_a74b725d47d149d497dcb57a8f4b8df0"
     }
    },
    "271083e34adf45bb8aab6b1e85193706": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e5109813dc714864851c06b46db9cae9",
      "max": 548105171,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_8234a4f00aef440c8ebd2069ff421853",
      "value": 548105171
     }
    },
    "27a4e299f86a47f6a77cb2fde6fd9607": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "280a539281a34d13878b1d678e40a160": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "29a292bd94c4456d8fd0eeb4973c0670": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "29d95cc429b74415b08672657ab320c7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "29e05722f6d94e049658cace58933099": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_e243128654cd4c44873240ecc4644a17",
       "IPY_MODEL_120b3e46872b41cdb2be157882b67ca4",
       "IPY_MODEL_9aba3a0e80c34c8790985250931253f1"
      ],
      "layout": "IPY_MODEL_2e61bcbdaa8d4ec299cc29ae92c80adb"
     }
    },
    "2a76cc1a0b46464bbd65fc92c4a64cc5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2a81d0cf6f654efeac3f2522b7fc0186": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2ac98ce69f164fdfbfb7ff99ddbec967": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "2d3e56d6572c45c7895b14699de36d87": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2a76cc1a0b46464bbd65fc92c4a64cc5",
      "placeholder": "​",
      "style": "IPY_MODEL_bf06dc9d10d247ec9971327d4dcc2b1a",
      "value": " 466k/466k [00:00&lt;00:00, 628kB/s]"
     }
    },
    "2daec0b4c2c941d2a2be950111102b2d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2db54785b3554d46811fde033a15dc17": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2de16307e11046418b98b90111b4009b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_9a78ed83139b4f26bf8282022e3c403e",
       "IPY_MODEL_046484ba77494b55acb7e13ed55a3487",
       "IPY_MODEL_12e1763b7a9b4ae78074cee4b5a409a3"
      ],
      "layout": "IPY_MODEL_47fddc1c2c074d87a6050b2a26d859f1"
     }
    },
    "2dfc9768d7a8421da53d9c812e7d9e8a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2e61bcbdaa8d4ec299cc29ae92c80adb": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2ee47a205c904782bdef418c25b89838": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2ef29cf73ea14794b8aaf9e3ccc78940": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2f488a3c842045d5ac7cc843f482427e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b3735b82dae8447faaca2f82af9d39c6",
      "max": 1042301,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_f244396283914d16baacaf5d6f76abac",
      "value": 1042301
     }
    },
    "305ec2194a844366910861a1c2b73201": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "311e3440f7a74343af959514ab41f35c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3133811b808e4fdfbcdaf81abb3a994e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "32945d26f3504f99a1b2140cd83d121e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "33268ee44c654b9e9154dae67b432ddc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_e9fe5354ad2749b6a3bb9d301a883f29",
       "IPY_MODEL_f145daed41fc44b3a65c58b81470c919",
       "IPY_MODEL_6c8aa9cbdfa64cba90c1c3ee782cf348"
      ],
      "layout": "IPY_MODEL_152381e5fe68427393a8a334a98e14a5"
     }
    },
    "33d1d6cd77c545d48c0cdc2a4207f05a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "340765c79f7b4b1fa7dc51f0db682e9c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "34ea08329bce40839467041d8d675b0b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_937d3dea8a2547f38287f8c61c8a3b8e",
      "max": 1289,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5e9280d23a81451da9d5898d9f5684b6",
      "value": 1289
     }
    },
    "3560a5a7dea44f0fb8b56a5f83fd5597": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "35f6d77e4970448088eef39b7fbec3e0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "37f56d81b5664fbe933e6c31826a37e4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f12b26bf7ecb4040b23d9d8c2fff7a6e",
      "placeholder": "​",
      "style": "IPY_MODEL_deb8982bab804d5c9fa3ba7627fad8d2",
      "value": "vocab.json: 100%"
     }
    },
    "3871c4f951d94bd1a86dc61312e3b6d0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_df66efedce50453f83d29acdd5f8b10e",
      "placeholder": "​",
      "style": "IPY_MODEL_441949f9bf114b879fb153334abb910b",
      "value": " 323/323 [00:00&lt;00:00, 3142.20 examples/s]"
     }
    },
    "387e47830d614fe8ba6758c7485a820c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8d567116324241c88d40d887ba44bbd1",
      "placeholder": "​",
      "style": "IPY_MODEL_edfecdad72b345c2ab68a674e3aee9ff",
      "value": "generation_config.json: 100%"
     }
    },
    "397a0d9b88b2483ab2736037772e97e8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3ac42e6d69fc4ea581d0fd3d1f77ea47": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_64fe242aa6184a7cbff04d4043267e05",
      "placeholder": "​",
      "style": "IPY_MODEL_a37527a86b78442f98e2591652d9075f",
      "value": " 483/483 [00:00&lt;00:00, 30.3kB/s]"
     }
    },
    "3b0e267772fa48fa87c6cc767885a590": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a647eaa49402408b8b10d8efb4c19bfd",
      "placeholder": "​",
      "style": "IPY_MODEL_4e505ed8ff2d4b62b139dbc7631f83bd",
      "value": " 1612/0 [00:00&lt;00:00, 18850.65 examples/s]"
     }
    },
    "3b2ed30f07e844388dae47f81d9ea65b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_960de534753949ada3cc727372f0db1d",
      "max": 665,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9e6171a639f544ceb56c091c57d9e619",
      "value": 665
     }
    },
    "3c290ca9b843494ea748504e4dd17655": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "3c42b1fb09cb44a2aef8cdd4c7961808": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_aa43590e7815414fbaea8accf7ae9297",
      "placeholder": "​",
      "style": "IPY_MODEL_90045b8b64324f729cb4e0873108546b",
      "value": "Map: 100%"
     }
    },
    "3c644b03c6ca4f388efbfe3f9cd573a4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3e08a44cdaee4ec18a767c85d61e89cb": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3e852e6825214df7bcda82aa091b55d1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_421997ecbc63432bbf12a049630dabab",
      "placeholder": "​",
      "style": "IPY_MODEL_08ef888fa03449dfbfed2c38b82b7008",
      "value": "config.json: 100%"
     }
    },
    "3f3fadc89d2d4144a814b69d511a5dd7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3f6daa12dce64d348ddd971c5417726e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "4086656b8cc54030b47a881760d396e4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "421997ecbc63432bbf12a049630dabab": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4262c25b50d945fbbac58cb13f2cf4aa": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_42f3c23f79804b588e05bfddb0442b09",
      "placeholder": "​",
      "style": "IPY_MODEL_ef3fc375222e4cf4a835fa898253cd8d",
      "value": "Map: 100%"
     }
    },
    "42d404e579f04988813895e1d0212f81": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "42dcbe8766f74de69749e3e0600e3829": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "42f3c23f79804b588e05bfddb0442b09": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4390ce9b054e4e44948a4afa1adfd214": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "439d37b4dbec4ee4afc0167555253edb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ea8505a10dee4b2f8e6d7ea9c3c30ed5",
       "IPY_MODEL_002a9501cdfb408a84a8779cf0176b02",
       "IPY_MODEL_62b8ba29be2c4518971e3ca4c03845c1"
      ],
      "layout": "IPY_MODEL_1f15b8d5547441cbb156b8dfce4ec7b0"
     }
    },
    "43bdb3e2257745268c506b8051afb03c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b1aba125897f4354a821bd5dd08ecd70",
      "placeholder": "​",
      "style": "IPY_MODEL_f571d7cf478b4c20a1f5c4bd503e7737",
      "value": " 771/771 [00:00&lt;00:00, 7677.94 examples/s]"
     }
    },
    "43e354bd826a4538a1bc08972f52f711": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "44050ee5b91145a8b22e2bc7de3ba85a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "441949f9bf114b879fb153334abb910b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "44c46ae2a0dd486e8b12b5a35f8ff0fa": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4665f471fa0e478882233b165c010b68": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "47989a32d3e54ede913cad2e881407ba": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_7273d05e25dd4af98ca58416bc2d1cb5",
       "IPY_MODEL_271083e34adf45bb8aab6b1e85193706",
       "IPY_MODEL_e5786c3017bf4a639a11f54f3aa7d9ce"
      ],
      "layout": "IPY_MODEL_51e8a4fe908c4a5b8895469ecfbe631b"
     }
    },
    "47cfdf980592498b893f8c3d2fbf0e03": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8cf83ceaa38044578713806efb84870a",
      "placeholder": "​",
      "style": "IPY_MODEL_311e3440f7a74343af959514ab41f35c",
      "value": "merges.txt: 100%"
     }
    },
    "47fddc1c2c074d87a6050b2a26d859f1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "484eaeabda91461093d8aac55b07b490": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c541c75959db47b89b78ef1e8ac06295",
      "max": 231508,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_0d25366a3d654d3fba85508bf309e568",
      "value": 231508
     }
    },
    "4866c0b43a964ac8bf21db1a2c76852a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_c06eb99f4ec241b9b010660294f63bb2",
       "IPY_MODEL_6b97eb69cc3d403abf0b32cd988f7706",
       "IPY_MODEL_9f7521e21b0d4b918bdefd2eb38fa75a"
      ],
      "layout": "IPY_MODEL_60635472c9f6460ab9d144e4c0a74271"
     }
    },
    "4969585bd7e842038c8f4f7d3bc6efe1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "49948cdf2ac94f45aa9c4a17c5996a17": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4a7dc41aa2824f918493115bda24498b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_387e47830d614fe8ba6758c7485a820c",
       "IPY_MODEL_85773884b394433b8f47703742a0413a",
       "IPY_MODEL_83f97c79b32d40f18c79d29974790dcf"
      ],
      "layout": "IPY_MODEL_340765c79f7b4b1fa7dc51f0db682e9c"
     }
    },
    "4b81c0892898485fafa62fb76028a133": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4bb8c462785b4ee3a905ef77c73a7a90": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_eb43725f0e22483989c5334facd46da0",
      "placeholder": "​",
      "style": "IPY_MODEL_2dfc9768d7a8421da53d9c812e7d9e8a",
      "value": " 232k/232k [00:00&lt;00:00, 484kB/s]"
     }
    },
    "4cac570ecf964757aec9d3a24d86138c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4d8e31a1dc0c4787891c15c6036ffab1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5e020a3384894ce9adffb1198a2eb181",
      "placeholder": "​",
      "style": "IPY_MODEL_5797df4963724a82821caf8d0b662274",
      "value": "Map: 100%"
     }
    },
    "4de0dcc6dfdb436abf0769cb0b2cf6e2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "4e07a852aa1b4cf1bb57b7c77515c133": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_afffdf5c259f4f14aeb99b6c25016ac6",
      "max": 466062,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_ff2247ba8c714949a5e80f599f4f47b4",
      "value": 466062
     }
    },
    "4e10af0f5cec42fb8cb3247af6430909": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4e505ed8ff2d4b62b139dbc7631f83bd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "4ec8c36c8d734fac9b41dfb2d123f5bd": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4edd5b00fbdb41ca8347c9884506837b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "4f792cc40ef34ae8a2b4a14d4538967e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "510e63d365c547b1adaff42d00490401": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "514fd68e776d4e4aaeb4a90f51bc4736": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "51abfd29aa04403e9a2b69eb42609b72": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_b32b1e7e3d1249cf8ec406631d9ae1a3",
       "IPY_MODEL_aa74920bb1ce4bdabd386ebe58f53abb",
       "IPY_MODEL_1bc31bdb12804834aa1e94ee7fa1b3ab"
      ],
      "layout": "IPY_MODEL_4ec8c36c8d734fac9b41dfb2d123f5bd"
     }
    },
    "51e8a4fe908c4a5b8895469ecfbe631b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "527d9fab58da45b1a9930c302c10d000": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_de96be39783b47c19fe9e3050ad3382d",
       "IPY_MODEL_484eaeabda91461093d8aac55b07b490",
       "IPY_MODEL_4bb8c462785b4ee3a905ef77c73a7a90"
      ],
      "layout": "IPY_MODEL_6b11860e26f94a3a8a71315334e4e3c6"
     }
    },
    "53a7239ac99e4fd889edf4be66091d35": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_32945d26f3504f99a1b2140cd83d121e",
      "placeholder": "​",
      "style": "IPY_MODEL_dbf509c4b21142ec9131647ab164374a",
      "value": "config.json: 100%"
     }
    },
    "53f0c26cedd14257988c3ea6cfb88cd0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "55dff7016e7a437ab8a732518f65ca4c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "57087b0cc5dd4a77be3beef53cb9a7d6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b21725fa1e684f4c92418af03f389a6f",
      "max": 629,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_e992d25a59fc4a71a9c1f02affe0e877",
      "value": 629
     }
    },
    "5797df4963724a82821caf8d0b662274": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "57b9fbf00fe34967b6f342524aae352b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c34e1baa25134764a8ecaa869b6bc406",
      "max": 570,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_305ec2194a844366910861a1c2b73201",
      "value": 570
     }
    },
    "57c90e7dadbe4879a2baaf4f22722f37": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "5923d06569814e5ba74e490eaf5c3c37": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_dbcdf8b2ea64486a8fb9b93aa29c546a",
      "placeholder": "​",
      "style": "IPY_MODEL_5ecabaa3f4904b52b9bfc6933ac4cd90",
      "value": " 456k/456k [00:00&lt;00:00, 9.96MB/s]"
     }
    },
    "5929f3be61e64cd3b03db83c6315f113": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_09e572cc5fa34a3dbfa76f898c90edab",
      "max": 665,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_c4381a91790446799969ce55f8a7250c",
      "value": 665
     }
    },
    "5a124cf7785e4eccafa4d1c7d3cd2942": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5a6a8a704b744c479bcabb9d5a16c8e8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_dddb342989e44ec0b0c9e40bb0ccf843",
      "placeholder": "​",
      "style": "IPY_MODEL_efb5452b364d4c358faea6a451f55bce",
      "value": "config.json: 100%"
     }
    },
    "5ac8c023798949aa85008138c59e5cbd": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5bf02116b4ae46389f0d60bf7f7b8816": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_77c90f063ce747aa9db15f2ae48d839d",
       "IPY_MODEL_9533899297ed48d1aa92583084412d9a",
       "IPY_MODEL_6b9430e328e246978f9162e256060cec"
      ],
      "layout": "IPY_MODEL_7e3f568ed5f5440e93379c6d5eb85a4b"
     }
    },
    "5bf3f7314dc74045aaa4cc1c4cd7bc7d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5d97ee42cb5b4ca5a9f5bc556f1a8a60": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "5dec607d9f0847c3be255d78656ef52b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_fae5f821c5604402af8f68f46ed4f2ca",
       "IPY_MODEL_5929f3be61e64cd3b03db83c6315f113",
       "IPY_MODEL_a8d497b06b8648fe8c06905fe9ebc973"
      ],
      "layout": "IPY_MODEL_a575be776efd4fa2be2bb978262efcfa"
     }
    },
    "5e020a3384894ce9adffb1198a2eb181": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5e05b1fd6b084563bc6b22e5650c6c79": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "5e9280d23a81451da9d5898d9f5684b6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "5ecabaa3f4904b52b9bfc6933ac4cd90": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5f174bf3a8d94b51be76f7455a6621f0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f123f91546064a538a74c83d893d52df",
      "placeholder": "​",
      "style": "IPY_MODEL_01ddfefb7f3e4a3589efab6d307a1bf0",
      "value": "Map: 100%"
     }
    },
    "5f9ee570b4034cfab50d5ce66f0f353f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6048025355d34b3ba0bcc2df379dfa05": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "60635472c9f6460ab9d144e4c0a74271": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "60784bb139dd474b86f3bd48c1a3103f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "608cdf8a656c4639ad7ed5bfbd77fab0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_44c46ae2a0dd486e8b12b5a35f8ff0fa",
      "placeholder": "​",
      "style": "IPY_MODEL_65fa9425c1634aa7983701ae7a6b4be3",
      "value": " 3081/3081 [00:00&lt;00:00, 8521.06 examples/s]"
     }
    },
    "60e6df2744cf40afaee1cbe071eae01e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "61d08e8218eb4c37a97a27ba42333517": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "61d1ee6617f3473da9b5e40b1ac00fbf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_072fea3dc9ba40f0b6e781f8755a322f",
      "placeholder": "​",
      "style": "IPY_MODEL_a1c3ad8b4f61434b9d41fb0ffda79991",
      "value": " 440M/440M [00:01&lt;00:00, 316MB/s]"
     }
    },
    "622af824740148f8a404ce3cf428db53": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "62b8ba29be2c4518971e3ca4c03845c1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9f1735288ea745f39f95127366bb8698",
      "placeholder": "​",
      "style": "IPY_MODEL_2a81d0cf6f654efeac3f2522b7fc0186",
      "value": " 48.0/48.0 [00:00&lt;00:00, 3.46kB/s]"
     }
    },
    "635101e9dd8a4448bc74a0ed4316c507": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a6e572e7ab67495b8a5b4b1a50af8e0b",
      "max": 771,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_ea7f12de56e446b19f407e6e092af4ba",
      "value": 771
     }
    },
    "6381a57dd82848689065de71dda55749": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "645f6ba5ef364830bfb4c43034233b18": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "64cecc3cdf97483990e52c2b0311601f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "64fe242aa6184a7cbff04d4043267e05": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "65a0252124e24f13a7db5e4c1bf29fc2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "65b52af198f04e0f8500eed14d3d4f8e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_43e354bd826a4538a1bc08972f52f711",
      "placeholder": "​",
      "style": "IPY_MODEL_aeea02bd03be406bb2b047e3e9b82f95",
      "value": "Map: 100%"
     }
    },
    "65fa5ad06c2841f4bf33c3d917dfe686": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_e9c3ea0435a14ce29d1c86b313ba9a54",
       "IPY_MODEL_d0465475c82e4484842703a19508ef59",
       "IPY_MODEL_cb4b2c83ca8a43e18823bb1c3a79fb0a"
      ],
      "layout": "IPY_MODEL_1f95a9be7f54444b88744c67e8da5a80"
     }
    },
    "65fa9425c1634aa7983701ae7a6b4be3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "666266a80a094b599299c3dffd219c36": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6772d51b7bb544eaa1c86e0836e49766": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "67c086e6a2e347ce8378a7ea4041b27d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_65b52af198f04e0f8500eed14d3d4f8e",
       "IPY_MODEL_635101e9dd8a4448bc74a0ed4316c507",
       "IPY_MODEL_43bdb3e2257745268c506b8051afb03c"
      ],
      "layout": "IPY_MODEL_e175619272a145949a2ec548819dd4eb"
     }
    },
    "67d215dd5e024d0d922c6ea2a8a1b36f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "687ec11348cd4044896fb407e3359a04": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "690943997d60421d8b948a0e0bf1e957": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ca388e55eafd41a483980574fb090a22",
      "placeholder": "​",
      "style": "IPY_MODEL_b816a4edc3bd41e0868a82e1d007e067",
      "value": " 26.0/26.0 [00:00&lt;00:00, 538B/s]"
     }
    },
    "6a07d0daadd84589abeca17d55fa0c8e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6a39130caf3f455d8f0f1d002a74bdb1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6a77df41417c4a84adaad20c94b23416": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6b11860e26f94a3a8a71315334e4e3c6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6b7101cea911496e98007d220c75a05b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0c2b3e3f7a494b6bb3e46003c6770571",
      "placeholder": "​",
      "style": "IPY_MODEL_3f6daa12dce64d348ddd971c5417726e",
      "value": "config.json: 100%"
     }
    },
    "6b9430e328e246978f9162e256060cec": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_29d95cc429b74415b08672657ab320c7",
      "placeholder": "​",
      "style": "IPY_MODEL_86875d1a86354e4787e872c98083210d",
      "value": " 1.04M/1.04M [00:00&lt;00:00, 4.25MB/s]"
     }
    },
    "6b97eb69cc3d403abf0b32cd988f7706": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4cac570ecf964757aec9d3a24d86138c",
      "max": 231508,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_2ac98ce69f164fdfbfb7ff99ddbec967",
      "value": 231508
     }
    },
    "6c8aa9cbdfa64cba90c1c3ee782cf348": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cdf02697fb654a65bcdf5886f102b4da",
      "placeholder": "​",
      "style": "IPY_MODEL_81304ddd745d4ed5be9b97220aa86095",
      "value": " 323/323 [00:00&lt;00:00, 6452.25 examples/s]"
     }
    },
    "6c9e2cc3d6b84d47bf71568457444be8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_da76d47c353f44068f5d2e41190e0d62",
      "placeholder": "​",
      "style": "IPY_MODEL_4969585bd7e842038c8f4f7d3bc6efe1",
      "value": " 771/771 [00:00&lt;00:00, 7082.60 examples/s]"
     }
    },
    "6d2a23d441284fc58af4502bd9c213f5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4d8e31a1dc0c4787891c15c6036ffab1",
       "IPY_MODEL_e1b5654d0d294ef9a0ac65bad7e41ad6",
       "IPY_MODEL_da6b104c271d4ea387af13b328e40fb6"
      ],
      "layout": "IPY_MODEL_2daec0b4c2c941d2a2be950111102b2d"
     }
    },
    "6d730e29f1794da08e4757a0bda3e67f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6f6b8a2799d9496a820168a6a33a62d5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a338d6cd66b144819521d0ab553719f1",
      "placeholder": "​",
      "style": "IPY_MODEL_9911f6d5447a410b9bf7a0e3dc966b13",
      "value": " 1.36M/1.36M [00:00&lt;00:00, 5.38MB/s]"
     }
    },
    "6fd602b801f0475db9c933fca5d2eb78": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7034dcd30e414f14a4c06c00cc2ade13": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6048025355d34b3ba0bcc2df379dfa05",
      "placeholder": "​",
      "style": "IPY_MODEL_6a77df41417c4a84adaad20c94b23416",
      "value": " 771/771 [00:08&lt;00:00, 69.60 examples/s]"
     }
    },
    "70ae6722b0d34f91b70f435b8162e878": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "70db9c1a40214b50a30862988d18e717": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "70e214355ffd434dacdd6b959e04b4c7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_8b1ed30ccf3143228a3aea84f390d41e",
       "IPY_MODEL_fc64554be0cc4dbcb7d17ef58fe55262",
       "IPY_MODEL_ba38d898cea04cc29db7a49d3d3367d4"
      ],
      "layout": "IPY_MODEL_abcd64d0a2ee4adeb749cc2fc34e3a6a"
     }
    },
    "7273d05e25dd4af98ca58416bc2d1cb5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b2414b78400c4a4c8a87cbbed9668d13",
      "placeholder": "​",
      "style": "IPY_MODEL_3560a5a7dea44f0fb8b56a5f83fd5597",
      "value": "model.safetensors: 100%"
     }
    },
    "739a81b3b3d44ac9b906227cad969bbc": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7742499693bb46a1888069358f2412a6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "77c90f063ce747aa9db15f2ae48d839d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cfbaf259829247f49994bf1e605b3e03",
      "placeholder": "​",
      "style": "IPY_MODEL_60e6df2744cf40afaee1cbe071eae01e",
      "value": "vocab.json: 100%"
     }
    },
    "784f4fb1e01241adab3acfcbf341a0ec": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "78b58c720fdd4720aa4bee7197007309": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "78f879c46a7f4a18ae2de70654121f0f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_01d0e06565c841cf8462f6d041873ad8",
      "max": 323,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_bf35bcd4037b4cfaa8554c210951ad29",
      "value": 323
     }
    },
    "79b3244660d64718b33e95af4bbef1d5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7a0c27699fcb4761a0f78253eb6af0ca": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7b53e5a3ecba425194221dab9273f08f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7c460ab96eed403cadba9519916c112b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7ce5227fb27046dfb9f47c48aa863fb8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7d9e3726eebd41f8b1d22aa80312df70": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_11a777252a044e3da149877541e3ecaf",
       "IPY_MODEL_e439a2c28d5c4ecf865619d7af82e698",
       "IPY_MODEL_0b601c2d8959449ba0eedeb6e858a6b3"
      ],
      "layout": "IPY_MODEL_9082dd5f5f2640a7a390a23684360ca9"
     }
    },
    "7e32b26c293e45b18eaa2e0c96ab145c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7e3f568ed5f5440e93379c6d5eb85a4b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7e9edc138e6840a4be81c100016ebc6f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7f8611121f774f5ea606cd968de34aa5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4f792cc40ef34ae8a2b4a14d4538967e",
      "placeholder": "​",
      "style": "IPY_MODEL_dc99ab2ea3704e0cb8279c142a738600",
      "value": " 629/629 [00:00&lt;00:00, 44.1kB/s]"
     }
    },
    "7fd547caf234435cb7952b211a538376": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7742499693bb46a1888069358f2412a6",
      "placeholder": "​",
      "style": "IPY_MODEL_5f9ee570b4034cfab50d5ce66f0f353f",
      "value": " 665/665 [00:00&lt;00:00, 53.2kB/s]"
     }
    },
    "804f844063e24173baf5e34af9cad49c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "81304ddd745d4ed5be9b97220aa86095": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8234a4f00aef440c8ebd2069ff421853": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "83f97c79b32d40f18c79d29974790dcf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b00cda330cfb40a9a35127f2cadf72d4",
      "placeholder": "​",
      "style": "IPY_MODEL_10a85b00fe6740bc8611137ac5bfc137",
      "value": " 124/124 [00:00&lt;00:00, 8.44kB/s]"
     }
    },
    "846546ae6e8649d5a93bb4012b58856d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "852ae3ffc3f945d1b5fbf3dc377ffc07": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "855ca149e747497f8a126585fe6abda7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_a5fe3f2998b44c4981f31f6c3070d7df",
       "IPY_MODEL_78f879c46a7f4a18ae2de70654121f0f",
       "IPY_MODEL_3871c4f951d94bd1a86dc61312e3b6d0"
      ],
      "layout": "IPY_MODEL_f99199d9122f49a0a36611c7e2e76419"
     }
    },
    "85773884b394433b8f47703742a0413a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_55dff7016e7a437ab8a732518f65ca4c",
      "max": 124,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_4de0dcc6dfdb436abf0769cb0b2cf6e2",
      "value": 124
     }
    },
    "85b192763f8a4480a3d4ae4e0957b685": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8630987be068488bbca15d661a8ba777": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "86875d1a86354e4787e872c98083210d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "86b098ea173a4419a9fe67d9c165b95b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "88445353976648f583ff1ec689f04d09": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7b53e5a3ecba425194221dab9273f08f",
      "max": 26,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_b6c7ff63249546808799c3b45711ede5",
      "value": 26
     }
    },
    "886bcbe9c1394d569f0c74fd7b54f0a5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "89325c9398474f9c92f16c63156d3038": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8a23f02ece4341f7900c82ab906874eb": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8a2d5a4ee0fd473b8137b518bfc3d68b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "8a916a0fe6874b7a97453b0f0dc314cf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "8aed6812b77f4175a7ba713d075b6db7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8b1d3a21ee4b4808847ea2a82a18ac61": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8b1ed30ccf3143228a3aea84f390d41e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4390ce9b054e4e44948a4afa1adfd214",
      "placeholder": "​",
      "style": "IPY_MODEL_f8e511c8d57a46a39ea1a67deabd6bfb",
      "value": "tokenizer_config.json: 100%"
     }
    },
    "8b7af1b3fb044d428f5a98dfb804a356": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8bd98770a54b40eba0468d1c7682c308": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8bfbfd341cc54418aaa9ca300aa6f54a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8cf83ceaa38044578713806efb84870a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8d567116324241c88d40d887ba44bbd1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8d9d4561ad3b4a6f865dadcfda4ac72a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8df7d56b3bb745caa64370e4c60e4205": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8e8a43aa4e11455484df7d141e6ad248": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c70600d1e4784ea7bf7e58b06ee47cc1",
      "placeholder": "​",
      "style": "IPY_MODEL_6381a57dd82848689065de71dda55749",
      "value": " 466k/466k [00:00&lt;00:00, 1.87MB/s]"
     }
    },
    "8f34f724a32e486da536cf0e55f3173c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "8ff6cd267a28487fb8cac4cb8724fc0e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "90045b8b64324f729cb4e0873108546b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "9026426dbb884fe39e2ec321a89954b2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9082dd5f5f2640a7a390a23684360ca9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "91d21b681ffb4ceaabd85f2867e60cbd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "925c24b98bec4b1ab62f4e2f3c5fc816": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4665f471fa0e478882233b165c010b68",
      "placeholder": "​",
      "style": "IPY_MODEL_53f0c26cedd14257988c3ea6cfb88cd0",
      "value": "tokenizer_config.json: 100%"
     }
    },
    "926dd4b6c57c42b68c44e06cb5013129": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ead4f1dddc0d48499955d55b069e82e0",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5d97ee42cb5b4ca5a9f5bc556f1a8a60",
      "value": 1
     }
    },
    "937d3dea8a2547f38287f8c61c8a3b8e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "938ab7319f314935afdab88c85d87894": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "94ef455f4eb940c0b176827e8a7d9fc5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "9533899297ed48d1aa92583084412d9a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_85b192763f8a4480a3d4ae4e0957b685",
      "max": 1042301,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_a186cd0d62634611941bbe875d707ab3",
      "value": 1042301
     }
    },
    "960de534753949ada3cc727372f0db1d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "975c4167faed4d339bbc3f2eeae9f748": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "979d607a3eee4671b0b93c5189bef09c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_e87270efd2594bbeb71d68f9f3f796f2",
       "IPY_MODEL_e4f5bea0b35742499724bcfe092bbaba",
       "IPY_MODEL_f0fefe6687614c32a11f90d91134f0e3"
      ],
      "layout": "IPY_MODEL_8630987be068488bbca15d661a8ba777"
     }
    },
    "98ecb94064cd4451b67cccdac5f8c102": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "98f94a4b641044809aa3bd2801fbee27": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_c30bd63cc2694712a657bc50f69046e8",
       "IPY_MODEL_aefcb034f9164e5ba707dd08c1e02a09",
       "IPY_MODEL_6c9e2cc3d6b84d47bf71568457444be8"
      ],
      "layout": "IPY_MODEL_33d1d6cd77c545d48c0cdc2a4207f05a"
     }
    },
    "99118a9b59864352ad3870d12b233d80": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_fbdf116a3aec45499f53c40a76b95741",
      "placeholder": "​",
      "style": "IPY_MODEL_af134db80bf741949c6c7248d921e874",
      "value": " 1.36M/1.36M [00:00&lt;00:00, 22.4MB/s]"
     }
    },
    "9911f6d5447a410b9bf7a0e3dc966b13": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "995ca1139e7f46fbac1c296c3dcda07a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9996784c4aa04ca480a58e53e5e073e5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a23cea85c11d44f99338ee370d27bb27",
      "placeholder": "​",
      "style": "IPY_MODEL_e4a5b2625bfc4f7eb9cf1000c8fcd9fb",
      "value": " 3852/0 [00:00&lt;00:00, 50546.27 examples/s]"
     }
    },
    "99b052cccf804bc3bbecb3d2a7b8eebb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_fcdff5e01eb7414c9325064a9be4f92a",
      "placeholder": "​",
      "style": "IPY_MODEL_ad5a96afd3bd478e9bb9e412d2770c9e",
      "value": "tokenizer.json: 100%"
     }
    },
    "9a78ed83139b4f26bf8282022e3c403e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_886bcbe9c1394d569f0c74fd7b54f0a5",
      "placeholder": "​",
      "style": "IPY_MODEL_42d404e579f04988813895e1d0212f81",
      "value": "tokenizer_config.json: 100%"
     }
    },
    "9aba3a0e80c34c8790985250931253f1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8b7af1b3fb044d428f5a98dfb804a356",
      "placeholder": "​",
      "style": "IPY_MODEL_0e0863081114481c964e57672c86c0b7",
      "value": " 232k/232k [00:00&lt;00:00, 471kB/s]"
     }
    },
    "9ad0006be39642f69cd73ee3bd5161d2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4e10af0f5cec42fb8cb3247af6430909",
      "max": 1355256,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_4edd5b00fbdb41ca8347c9884506837b",
      "value": 1355256
     }
    },
    "9ba88a2e42b54a59a6aa9907e5c57fa0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8a2d5a4ee0fd473b8137b518bfc3d68b",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_67d215dd5e024d0d922c6ea2a8a1b36f",
      "value": 1
     }
    },
    "9e6171a639f544ceb56c091c57d9e619": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "9e9badec71f74fdf997043d9a628c597": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_3c42b1fb09cb44a2aef8cdd4c7961808",
       "IPY_MODEL_c2c41e586b5d47aa93ba8e4dc8675a32",
       "IPY_MODEL_7034dcd30e414f14a4c06c00cc2ade13"
      ],
      "layout": "IPY_MODEL_2ef29cf73ea14794b8aaf9e3ccc78940"
     }
    },
    "9ec4bf7395dc42ea8022959062f6f333": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_bfdf1f0d388942beb50bcd7242bdb3df",
       "IPY_MODEL_34ea08329bce40839467041d8d675b0b",
       "IPY_MODEL_9ef31f7bcb7342108ac855ceb0d76b49"
      ],
      "layout": "IPY_MODEL_f2c83ee98d4d44389e9354b89a914cd1"
     }
    },
    "9ef31f7bcb7342108ac855ceb0d76b49": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ea6dca3ac29d409daa542899e8b7a3e1",
      "placeholder": "​",
      "style": "IPY_MODEL_8bfbfd341cc54418aaa9ca300aa6f54a",
      "value": " 1289/1289 [00:00&lt;00:00, 6410.19 examples/s]"
     }
    },
    "9f1735288ea745f39f95127366bb8698": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9f3d7cd9902149938997025d735d83ab": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_37f56d81b5664fbe933e6c31826a37e4",
       "IPY_MODEL_2f488a3c842045d5ac7cc843f482427e",
       "IPY_MODEL_b3e45aff192641afa35642453884a8e1"
      ],
      "layout": "IPY_MODEL_a726e761ed344feb90b0058cf65498e7"
     }
    },
    "9f7521e21b0d4b918bdefd2eb38fa75a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_181ee900eccb4bdc98945cae0e95b770",
      "placeholder": "​",
      "style": "IPY_MODEL_120fbeddc7d3406994260af25c636e5e",
      "value": " 232k/232k [00:00&lt;00:00, 942kB/s]"
     }
    },
    "9fcc4a2ca21042848e5becbb152cb681": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a12d5562a5d84bca8378845a92411bc6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a1754c0bf4d9463ca5214eac4eb9a1eb": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a186cd0d62634611941bbe875d707ab3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "a1c3ad8b4f61434b9d41fb0ffda79991": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a23cea85c11d44f99338ee370d27bb27": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a338d6cd66b144819521d0ab553719f1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a37527a86b78442f98e2591652d9075f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a40ffd8b51784c2988e75903ac4fab93": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_fc19e87c5a9b409e8877022912b3b5d4",
      "placeholder": "​",
      "style": "IPY_MODEL_d747d7a4a9ac4e479e2cafb7ca11d0ad",
      "value": " 3852/3852 [00:00&lt;00:00, 19595.11 examples/s]"
     }
    },
    "a575be776efd4fa2be2bb978262efcfa": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a5fe3f2998b44c4981f31f6c3070d7df": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a747ee6a682d4888a16451b2fe57967d",
      "placeholder": "​",
      "style": "IPY_MODEL_b47d7ef257c449f1b0e8756446997d89",
      "value": "Map: 100%"
     }
    },
    "a647eaa49402408b8b10d8efb4c19bfd": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a6e572e7ab67495b8a5b4b1a50af8e0b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a726e761ed344feb90b0058cf65498e7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a747ee6a682d4888a16451b2fe57967d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a74b725d47d149d497dcb57a8f4b8df0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a7c1ffd3bfae4ac29997b77f805d6684": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a8d497b06b8648fe8c06905fe9ebc973": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7c460ab96eed403cadba9519916c112b",
      "placeholder": "​",
      "style": "IPY_MODEL_ba8a10a3495d4a5b87e65562d115f7c5",
      "value": " 665/665 [00:00&lt;00:00, 26.4kB/s]"
     }
    },
    "aa43590e7815414fbaea8accf7ae9297": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "aa74920bb1ce4bdabd386ebe58f53abb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_846546ae6e8649d5a93bb4012b58856d",
      "max": 3081,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_b080efa6d7a34395bc08542164b2b6d4",
      "value": 3081
     }
    },
    "abaa724f6e984ddcb538a24012a74dd9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "abcd64d0a2ee4adeb749cc2fc34e3a6a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ad3da69addc44dcd910c982aabd94fb5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6d730e29f1794da08e4757a0bda3e67f",
      "max": 26,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_622af824740148f8a404ce3cf428db53",
      "value": 26
     }
    },
    "ad5a96afd3bd478e9bb9e412d2770c9e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "adaaaa75b6f047239329f6a74b7c2f77": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3133811b808e4fdfbcdaf81abb3a994e",
      "placeholder": "​",
      "style": "IPY_MODEL_0775a4d8fe1d4cd3b34486bf28d5f755",
      "value": " 456k/456k [00:00&lt;00:00, 6.10MB/s]"
     }
    },
    "aeea02bd03be406bb2b047e3e9b82f95": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "aefcb034f9164e5ba707dd08c1e02a09": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_13192bc720d84c2cb83a4cca9906471a",
      "max": 771,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_94ef455f4eb940c0b176827e8a7d9fc5",
      "value": 771
     }
    },
    "af134db80bf741949c6c7248d921e874": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "afffdf5c259f4f14aeb99b6c25016ac6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b00cda330cfb40a9a35127f2cadf72d4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b036589813fb4676b956e89e846d4362": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b080efa6d7a34395bc08542164b2b6d4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "b1aba125897f4354a821bd5dd08ecd70": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b21725fa1e684f4c92418af03f389a6f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b2414b78400c4a4c8a87cbbed9668d13": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b2610c7ea73e437083cd43ef671cc563": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b32b1e7e3d1249cf8ec406631d9ae1a3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7a0c27699fcb4761a0f78253eb6af0ca",
      "placeholder": "​",
      "style": "IPY_MODEL_0db028ff95e84d33acdb533760ad41da",
      "value": "Map: 100%"
     }
    },
    "b3735b82dae8447faaca2f82af9d39c6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b3e45aff192641afa35642453884a8e1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_da491b6e5fde4debbfb30f5433848a60",
      "placeholder": "​",
      "style": "IPY_MODEL_0d750e71cfd04203bc6245d72e62cdc5",
      "value": " 1.04M/1.04M [00:00&lt;00:00, 4.22MB/s]"
     }
    },
    "b43bdb8595674ff9b9cd369c2edeea9f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "b47d7ef257c449f1b0e8756446997d89": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b53346bd908147038ebfb4b164580c10": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_035cec73334d405d860369ff47340d4d",
       "IPY_MODEL_c55f4a29cd6b46e285f2b3b26e34ff17",
       "IPY_MODEL_5923d06569814e5ba74e490eaf5c3c37"
      ],
      "layout": "IPY_MODEL_804f844063e24173baf5e34af9cad49c"
     }
    },
    "b6c7ff63249546808799c3b45711ede5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "b7d267442de04961a6f52c38d00c2edc": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b816a4edc3bd41e0868a82e1d007e067": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b89907d757a74899a7606c3b6ce383e5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_49948cdf2ac94f45aa9c4a17c5996a17",
      "placeholder": "​",
      "style": "IPY_MODEL_0c78bace44154985ad12c0265ae430f7",
      "value": "Generating train split: "
     }
    },
    "ba38d898cea04cc29db7a49d3d3367d4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d1b6a2821d0d42069d6cf483eb634d0b",
      "placeholder": "​",
      "style": "IPY_MODEL_10570d170bec405c8c15d80523a0717c",
      "value": " 28.0/28.0 [00:00&lt;00:00, 1.80kB/s]"
     }
    },
    "ba8a10a3495d4a5b87e65562d115f7c5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "bbb94e68c17e423f8dd9fc1a9438aa94": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "bd1e564382c4439c82645445d421ecbf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8d9d4561ad3b4a6f865dadcfda4ac72a",
      "max": 1355256,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_8f34f724a32e486da536cf0e55f3173c",
      "value": 1355256
     }
    },
    "be8fcd3bd159483081ab057dcc861aa2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bf06dc9d10d247ec9971327d4dcc2b1a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "bf35bcd4037b4cfaa8554c210951ad29": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "bfdf1f0d388942beb50bcd7242bdb3df": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d2a05abfd84e402d811f9a976d724478",
      "placeholder": "​",
      "style": "IPY_MODEL_03ffc521b0874dc6aeb0beb2cefae358",
      "value": "Map: 100%"
     }
    },
    "c06eb99f4ec241b9b010660294f63bb2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_645f6ba5ef364830bfb4c43034233b18",
      "placeholder": "​",
      "style": "IPY_MODEL_14e98cabb95c438abf13993167e4618f",
      "value": "vocab.txt: 100%"
     }
    },
    "c11c4c4acddf4a3cb62fcd72b50860d4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c2c41e586b5d47aa93ba8e4dc8675a32": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7e9edc138e6840a4be81c100016ebc6f",
      "max": 771,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_16514425790a4386ba23ea1ff1cdda3b",
      "value": 771
     }
    },
    "c30bd63cc2694712a657bc50f69046e8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6a07d0daadd84589abeca17d55fa0c8e",
      "placeholder": "​",
      "style": "IPY_MODEL_938ab7319f314935afdab88c85d87894",
      "value": "Map: 100%"
     }
    },
    "c34e1baa25134764a8ecaa869b6bc406": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c4381a91790446799969ce55f8a7250c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "c43c28578be8437ab5457b444b4103fd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "c541c75959db47b89b78ef1e8ac06295": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c55f4a29cd6b46e285f2b3b26e34ff17": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d5cc7296b9714a448e44384bf8f62bf3",
      "max": 456318,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_0392f7a3461842f3bc9ef9be05242229",
      "value": 456318
     }
    },
    "c5c9de8c50f94005905df2e4d0242dc5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5a6a8a704b744c479bcabb9d5a16c8e8",
       "IPY_MODEL_03b0d4b80ee8400e8bb1ff96fe794139",
       "IPY_MODEL_3ac42e6d69fc4ea581d0fd3d1f77ea47"
      ],
      "layout": "IPY_MODEL_995ca1139e7f46fbac1c296c3dcda07a"
     }
    },
    "c60c89940e5e455bab9c7cf1e420d55b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5f174bf3a8d94b51be76f7455a6621f0",
       "IPY_MODEL_d0db897511f0440588131cd7621ab41b",
       "IPY_MODEL_a40ffd8b51784c2988e75903ac4fab93"
      ],
      "layout": "IPY_MODEL_5ac8c023798949aa85008138c59e5cbd"
     }
    },
    "c61dc8ce463444d1909d23cd6071b355": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c70600d1e4784ea7bf7e58b06ee47cc1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ca388e55eafd41a483980574fb090a22": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cb4b2c83ca8a43e18823bb1c3a79fb0a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e40b00b93fe14e5ba62613d5fa74acf8",
      "placeholder": "​",
      "style": "IPY_MODEL_063a969f176941fa84a21569b4c6ba2b",
      "value": " 3081/3081 [00:00&lt;00:00, 10092.86 examples/s]"
     }
    },
    "cb7fa84b89ca428f9004e14647b8c876": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ed9a7513f10e4bf390e853a725541922",
      "placeholder": "​",
      "style": "IPY_MODEL_79b3244660d64718b33e95af4bbef1d5",
      "value": " 268M/268M [00:06&lt;00:00, 40.4MB/s]"
     }
    },
    "cba3e1f715b24d03b8cf179efb41055b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "cc9a28d85306494ea73be4bbe52e9d17": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cd067f60eb7c45bf8010431d51b2fb34": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_b89907d757a74899a7606c3b6ce383e5",
       "IPY_MODEL_926dd4b6c57c42b68c44e06cb5013129",
       "IPY_MODEL_9996784c4aa04ca480a58e53e5e073e5"
      ],
      "layout": "IPY_MODEL_abaa724f6e984ddcb538a24012a74dd9"
     }
    },
    "cd47ce9df8cf4d4aae89672fc0602b71": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cdf02697fb654a65bcdf5886f102b4da": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ce749881b13f4db38ccbc2d8a3fb09cd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "cfbaf259829247f49994bf1e605b3e03": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d0465475c82e4484842703a19508ef59": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ee8c3960cb684a54adc10a0516f3fc75",
      "max": 3081,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_8a916a0fe6874b7a97453b0f0dc314cf",
      "value": 3081
     }
    },
    "d0786942e0df48e78edc03829aa53a0c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d0db897511f0440588131cd7621ab41b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8aed6812b77f4175a7ba713d075b6db7",
      "max": 3852,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_c43c28578be8437ab5457b444b4103fd",
      "value": 3852
     }
    },
    "d1b6a2821d0d42069d6cf483eb634d0b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d2a05abfd84e402d811f9a976d724478": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d3a7c090b74f43c7bc2b6008ec0f44ae": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d4ce47e5a2d34f4488171fd928918db3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4086656b8cc54030b47a881760d396e4",
      "max": 548105171,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_b43bdb8595674ff9b9cd369c2edeea9f",
      "value": 548105171
     }
    },
    "d5cc7296b9714a448e44384bf8f62bf3": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d62f299a59d04f4da693b6524a501a2e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4262c25b50d945fbbac58cb13f2cf4aa",
       "IPY_MODEL_19f76eaac0a74742adc27e9cdad38b65",
       "IPY_MODEL_608cdf8a656c4639ad7ed5bfbd77fab0"
      ],
      "layout": "IPY_MODEL_e268bdded7af4cd39467d93be5b4621d"
     }
    },
    "d747d7a4a9ac4e479e2cafb7ca11d0ad": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d847ba4af98546978dafacf9e9b0fc95": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_99b052cccf804bc3bbecb3d2a7b8eebb",
       "IPY_MODEL_bd1e564382c4439c82645445d421ecbf",
       "IPY_MODEL_6f6b8a2799d9496a820168a6a33a62d5"
      ],
      "layout": "IPY_MODEL_7e32b26c293e45b18eaa2e0c96ab145c"
     }
    },
    "d9cdf9667f6746f382f5f10f772b3144": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "da491b6e5fde4debbfb30f5433848a60": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "da6b104c271d4ea387af13b328e40fb6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8df7d56b3bb745caa64370e4c60e4205",
      "placeholder": "​",
      "style": "IPY_MODEL_5bf3f7314dc74045aaa4cc1c4cd7bc7d",
      "value": " 1289/1289 [00:00&lt;00:00, 3293.38 examples/s]"
     }
    },
    "da76d47c353f44068f5d2e41190e0d62": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "da814b1528be4926975893b54fd72849": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "db17d88724f845568a4396b9a5ce508a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_eb8c530d239d4a4bb39f0296e67dc76d",
      "max": 456318,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_510e63d365c547b1adaff42d00490401",
      "value": 456318
     }
    },
    "db1d3dc2171b4910a5e07ba1e7d5f734": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dbcdf8b2ea64486a8fb9b93aa29c546a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dbf509c4b21142ec9131647ab164374a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "dc87398fce094a658da749f69feb4fc8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cc9a28d85306494ea73be4bbe52e9d17",
      "placeholder": "​",
      "style": "IPY_MODEL_cba3e1f715b24d03b8cf179efb41055b",
      "value": "tokenizer.json: 100%"
     }
    },
    "dc99ab2ea3704e0cb8279c142a738600": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "dd8a687326dc49feafc45355eb129d95": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_19a2336a406a45bf9e990fa8ad9a49d4",
       "IPY_MODEL_13a2a27f04f7451aa8be99bb139efb27",
       "IPY_MODEL_2d3e56d6572c45c7895b14699de36d87"
      ],
      "layout": "IPY_MODEL_0d27001bb0cb4b689a75fbea7f1efa4c"
     }
    },
    "dddb342989e44ec0b0c9e40bb0ccf843": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ddf9c083e48f415d92d2ce2322af149a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "de96be39783b47c19fe9e3050ad3382d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8a23f02ece4341f7900c82ab906874eb",
      "placeholder": "​",
      "style": "IPY_MODEL_91d21b681ffb4ceaabd85f2867e60cbd",
      "value": "vocab.txt: 100%"
     }
    },
    "deb8982bab804d5c9fa3ba7627fad8d2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "df66efedce50453f83d29acdd5f8b10e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e175619272a145949a2ec548819dd4eb": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e1b5654d0d294ef9a0ac65bad7e41ad6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8b1d3a21ee4b4808847ea2a82a18ac61",
      "max": 1289,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_57c90e7dadbe4879a2baaf4f22722f37",
      "value": 1289
     }
    },
    "e243128654cd4c44873240ecc4644a17": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_14d54788f46d479eb0faa38af384cdc8",
      "placeholder": "​",
      "style": "IPY_MODEL_a12d5562a5d84bca8378845a92411bc6",
      "value": "vocab.txt: 100%"
     }
    },
    "e268bdded7af4cd39467d93be5b4621d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e3941149c2a84b6ab6f61b6666e393af": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "e40b00b93fe14e5ba62613d5fa74acf8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e439a2c28d5c4ecf865619d7af82e698": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a1754c0bf4d9463ca5214eac4eb9a1eb",
      "max": 267954768,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_106cc09c16f84c419d0d2cd5c7f55a8a",
      "value": 267954768
     }
    },
    "e4a5b2625bfc4f7eb9cf1000c8fcd9fb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e4db92d02bbd416daed51e431353de58": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_be8fcd3bd159483081ab057dcc861aa2",
      "placeholder": "​",
      "style": "IPY_MODEL_397a0d9b88b2483ab2736037772e97e8",
      "value": "tokenizer.json: 100%"
     }
    },
    "e4f5bea0b35742499724bcfe092bbaba": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c11c4c4acddf4a3cb62fcd72b50860d4",
      "max": 124,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_3c290ca9b843494ea748504e4dd17655",
      "value": 124
     }
    },
    "e5109813dc714864851c06b46db9cae9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e5786c3017bf4a639a11f54f3aa7d9ce": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cd47ce9df8cf4d4aae89672fc0602b71",
      "placeholder": "​",
      "style": "IPY_MODEL_d0786942e0df48e78edc03829aa53a0c",
      "value": " 548M/548M [00:01&lt;00:00, 325MB/s]"
     }
    },
    "e6f9a976e59e42618a3c511f427acb88": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ed7ba59f9d504f6c841ebcd8386efdb9",
      "placeholder": "​",
      "style": "IPY_MODEL_6a39130caf3f455d8f0f1d002a74bdb1",
      "value": " 570/570 [00:00&lt;00:00, 43.5kB/s]"
     }
    },
    "e87270efd2594bbeb71d68f9f3f796f2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b036589813fb4676b956e89e846d4362",
      "placeholder": "​",
      "style": "IPY_MODEL_687ec11348cd4044896fb407e3359a04",
      "value": "generation_config.json: 100%"
     }
    },
    "e9164b241b124b83accf07de6272cf0b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e992d25a59fc4a71a9c1f02affe0e877": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "e9c3ea0435a14ce29d1c86b313ba9a54": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ee79ca16bfe24e658376c11ddfda1124",
      "placeholder": "​",
      "style": "IPY_MODEL_70ae6722b0d34f91b70f435b8162e878",
      "value": "Map: 100%"
     }
    },
    "e9fe5354ad2749b6a3bb9d301a883f29": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_60784bb139dd474b86f3bd48c1a3103f",
      "placeholder": "​",
      "style": "IPY_MODEL_0e366bd39b2e4bb78bf145447a329555",
      "value": "Map: 100%"
     }
    },
    "ea6dca3ac29d409daa542899e8b7a3e1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ea7f12de56e446b19f407e6e092af4ba": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "ea8505a10dee4b2f8e6d7ea9c3c30ed5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_35f6d77e4970448088eef39b7fbec3e0",
      "placeholder": "​",
      "style": "IPY_MODEL_ddf9c083e48f415d92d2ce2322af149a",
      "value": "tokenizer_config.json: 100%"
     }
    },
    "ead4f1dddc0d48499955d55b069e82e0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "eb0b856aedb744b9831994bdd921d705": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "eb43725f0e22483989c5334facd46da0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "eb8c530d239d4a4bb39f0296e67dc76d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ebacdfc661154ae386dd3ccaeb23f772": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ec120f1e3ebd44f1bc98d8fd7bfecc14": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_86b098ea173a4419a9fe67d9c165b95b",
      "placeholder": "​",
      "style": "IPY_MODEL_02c2e1530b63441e9954920f3c27a606",
      "value": "Generating train split: "
     }
    },
    "ed7ba59f9d504f6c841ebcd8386efdb9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ed9a7513f10e4bf390e853a725541922": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "edf1b0f7c7684e08a817590e9a201eb4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_efde0db3e6914b7f950b0a457bbcb4ae",
      "placeholder": "​",
      "style": "IPY_MODEL_ee741717c13740faaf795193f91cbb6b",
      "value": "model.safetensors: 100%"
     }
    },
    "edfecdad72b345c2ab68a674e3aee9ff": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ee741717c13740faaf795193f91cbb6b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ee79ca16bfe24e658376c11ddfda1124": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ee8c3960cb684a54adc10a0516f3fc75": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "eed9340ed9d04611b71f75e4db6cd67e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ef3fc375222e4cf4a835fa898253cd8d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "efb5452b364d4c358faea6a451f55bce": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "efde0db3e6914b7f950b0a457bbcb4ae": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f0fefe6687614c32a11f90d91134f0e3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_da814b1528be4926975893b54fd72849",
      "placeholder": "​",
      "style": "IPY_MODEL_ce749881b13f4db38ccbc2d8a3fb09cd",
      "value": " 124/124 [00:00&lt;00:00, 3.39kB/s]"
     }
    },
    "f123f91546064a538a74c83d893d52df": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f12b26bf7ecb4040b23d9d8c2fff7a6e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f145daed41fc44b3a65c58b81470c919": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_eb0b856aedb744b9831994bdd921d705",
      "max": 323,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_65a0252124e24f13a7db5e4c1bf29fc2",
      "value": 323
     }
    },
    "f21888da6c744c68994334969d6656cb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f244396283914d16baacaf5d6f76abac": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "f28cbac2858e4e079790d7f24ff1d73d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f2a9bf4b0fec4fb6a0942174e2198046": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ec120f1e3ebd44f1bc98d8fd7bfecc14",
       "IPY_MODEL_9ba88a2e42b54a59a6aa9907e5c57fa0",
       "IPY_MODEL_3b0e267772fa48fa87c6cc767885a590"
      ],
      "layout": "IPY_MODEL_3e08a44cdaee4ec18a767c85d61e89cb"
     }
    },
    "f2c83ee98d4d44389e9354b89a914cd1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f4574e12d5db48ca862bfb2811fad258": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_12afd487ae7343f7b8a65aaa60bd00c7",
       "IPY_MODEL_d4ce47e5a2d34f4488171fd928918db3",
       "IPY_MODEL_244018be291643fe9e929eaf3e531098"
      ],
      "layout": "IPY_MODEL_f6e0e64e2c234c559c8160f06b28b618"
     }
    },
    "f571d7cf478b4c20a1f5c4bd503e7737": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f601a3994de84cc9885036e8d1c5a8ee": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f6e0e64e2c234c559c8160f06b28b618": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f8e511c8d57a46a39ea1a67deabd6bfb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f99199d9122f49a0a36611c7e2e76419": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f9971e233de04afb9a9d4e6e2591d729": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_925c24b98bec4b1ab62f4e2f3c5fc816",
       "IPY_MODEL_ad3da69addc44dcd910c982aabd94fb5",
       "IPY_MODEL_0bdfa1d6d98d42069dfd65dba4889c8e"
      ],
      "layout": "IPY_MODEL_c61dc8ce463444d1909d23cd6071b355"
     }
    },
    "fae5f821c5604402af8f68f46ed4f2ca": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_739a81b3b3d44ac9b906227cad969bbc",
      "placeholder": "​",
      "style": "IPY_MODEL_d3a7c090b74f43c7bc2b6008ec0f44ae",
      "value": "config.json: 100%"
     }
    },
    "fb82f5287b4a4a8c8e2ede1838c1dbeb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_6b7101cea911496e98007d220c75a05b",
       "IPY_MODEL_57087b0cc5dd4a77be3beef53cb9a7d6",
       "IPY_MODEL_7f8611121f774f5ea606cd968de34aa5"
      ],
      "layout": "IPY_MODEL_e9164b241b124b83accf07de6272cf0b"
     }
    },
    "fbdf116a3aec45499f53c40a76b95741": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fc19e87c5a9b409e8877022912b3b5d4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fc64554be0cc4dbcb7d17ef58fe55262": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2ee47a205c904782bdef418c25b89838",
      "max": 28,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_8ff6cd267a28487fb8cac4cb8724fc0e",
      "value": 28
     }
    },
    "fcdff5e01eb7414c9325064a9be4f92a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fce0f472357142ecaf627eeeee9c2e75": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "ff2247ba8c714949a5e80f599f4f47b4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "ff804399b8e64b4ea6795b5df7ac99ea": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1dff6d902ca749d29775a39b53e98e03",
      "placeholder": "​",
      "style": "IPY_MODEL_f21888da6c744c68994334969d6656cb",
      "value": "tokenizer_config.json: 100%"
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
